Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
33f15d9ce0 | ||
|
|
09af3bd8c0 | ||
|
|
5977b1b272 | ||
|
|
97e84fa6c4 | ||
|
|
173c7a042d | ||
|
|
a4179abb06 | ||
|
|
fe34ccc38a | ||
|
|
178a8f5e19 | ||
|
|
03de63f48f | ||
|
|
99c59d443d |
No files matched your search
|
After Width: | Height: | Size: 52 KiB |
|
After Width: | Height: | Size: 120 KiB |
|
After Width: | Height: | Size: 116 KiB |
|
After Width: | Height: | Size: 172 KiB |
|
After Width: | Height: | Size: 131 KiB |
@@ -1,14 +1,13 @@
|
||||
# 工厂通讯:一个发展巴布石金式工人同志的案例
|
||||
## 编者按:
|
||||
|
||||
编者按:
|
||||
同志A完全信任我们的工人同志,我们认为主要源于这位同志长时间的沟通,以及他偶然注意到A的留言动态,通过内容来判断,由此进行关心。在这个基础上,我们有充分的理由相信,同志在做思想工作的过程中,应当是没有架子,做到了群众化的。在A的视角,我们的同志只是具备理论知识的打工人,他们之间是可以“随意”交流的,在他们的沟通和相处中,我们的同志取得了A的信任。为什么他能够得到信任呢?在客观上,是A的部分改良主义思想破产及资产阶级爱情观的打击,在主观上,我们的同志及时注意到了同志A的情绪低落状态,然后在这个节点,采用了解和关心的方式方法,首先在情感上取得了A的信任,在这个基础上,逐渐从情绪或者说从感性的层面,逐步发展为理性的灌输阶段。
|
||||
并且同志A在现实中遇到的各种问题,我们的同志也可以解释并很好地回应,这反映出在对待同志A的工作中,同志的方式方法充分得到了现实的结果支撑。总之,这次成果主要来源于对现有感性材料的合理判断,加上同志的主观努力。这也说明了具体的灌输工作的曲折性:在不同的工人之间,这种灌输往往并不是直线的,所以需要注意保持恒心,善于发现工友关注的矛盾的点,并进行针对性的交流,这点需要我们在实际工作中不断锻炼。
|
||||
|
||||
说明:
|
||||
>
|
||||
> - 我 - 厂内同志
|
||||
> - 同志A - 我本地发展的工人同志
|
||||
> 之所以开始写这个问题,是因为我这边发展的同志A的工作已经达到了较好的水平,可以做阶段汇报了。
|
||||
- 我 - 厂内同志
|
||||
- 同志A - 我本地发展的工人同志
|
||||
|
||||
之所以开始写这个问题,是因为我这边发展的同志A的工作已经达到了较好的水平,可以做阶段汇报了。
|
||||
|
||||
## 一、当前的工作进展
|
||||
### 同志A已能够独立发展新同志
|
||||
@@ -66,7 +65,7 @@
|
||||
|
||||
他喜欢说在保安岗位上怎么不听老板的话,以及故意把车放走不收停车费让他们损失。但对导致自己离职或可能有财产损失的斗争就犹豫。我问他,他爱谈的那些斗争对于别人有什么帮助呢?他如果能通过斗争,使得他即使早走也能拿到工资,那以后也能教别人怎么做,这样才会对别人有帮助。实际上这就是因为他还是太顾自己,不够考虑别人,另外也不够考虑政治,只是考虑这点钱,为了这点钱而一直被他们拖着浪费时间。
|
||||
|
||||
3. 个人经历对革命路线的影响
|
||||
### 3. 个人经历对革命路线的影响
|
||||
**应该结合他生产上的地位来分析这些路线**,这位同志之前在工厂里做过,厂里有次几个人在厕所抽烟被抓了,本来说要罚款,大家都不同意,只好作罢。他也觉得厂里的人会比别的地方的更可能反抗。他说要是厂里人都罢工,资本家是没办法的,他只能求你动一下、动一下,工人就可以提出自己的诉求。不过他因为搞物业当保安,并且之前还和一些流氓无产阶级有交往,认识的人很杂。唯军事论、缺乏政治挂帅的路线就是来源于流氓无产阶级的影响,还有保安这种非生产的行业也会有无政府主义的倾向。
|
||||
|
||||
**他在这些经历中不可避免地会形成这种思想,就需要结合他在厂里受到过的好的影响去与其他经历中一些不好的影响做斗争,以及去改造他的社会关系。比如他认识的流氓无产阶级,我接触他以后就让他断掉了**,和他说那种人和资本家实际上是一个硬币的两面,他也能理解。我们应该有这种自信:我们能把新同志拉到正确的路线上。他说我是他遇到的对他最好的人,他自己会比较冲动,而我比较理智,像他的军师一样,他愿意听我的意见。
|
||||
|
||||
@@ -135,7 +135,7 @@ _思考:_
|
||||
|
||||
- 在提提前面说的护士与我们这些实习生交流的,年轻的护士很多会与我们有共鸣,***除了他们本身工资不高以外,就是面对奇奇怪怪的患者使得她们很崩溃,处理各种就是与她们自身专业不是很相关的事物以及其他的东西***,她们会觉得自己的工资无法匹配这个工作强度,特别是病人居多的科室,这意味输液病人很多,再加上文书什么的,非常忙碌,我们这些实习生同样也是连轴转,还有就是老护士对新护士的压迫,很多时候特别是换输液瓶的时候,都是老护士让新护士去或者让实习生去,或者是做其他的杂活什么的,如果不去的话,可能就是会什么故意针对以及直接找护士长告状诸如此类的,所以说我们实习生和新的年轻护士之间讨论就可以放的开了。
|
||||
|
||||
> 最后的最后,如果让我和她们说马列毛的相关内容,就我那时候的情况而言,可能讲不了多少,不过我个人觉得应该还是说我要与她们的关系更熟悉,然后我自己也能把这种马列毛的分析融入到里面告诉她们,目前我也不能确定有没有其他更好的办法了。
|
||||
- 最后的最后,如果让我和她们说马列毛的相关内容,就我那时候的情况而言,可能讲不了多少,不过我个人觉得应该还是说我要与她们的关系更熟悉,然后我自己也能把这种马列毛的分析融入到里面告诉她们,目前我也不能确定有没有其他更好的办法了。
|
||||
|
||||
|
||||
另:
|
||||
|
||||
@@ -13,11 +13,8 @@
|
||||
在这一期中,工革报好不容易写出来了一个路线式的段落。我们把这一段落摘录如下:
|
||||
|
||||
> 工革报编辑部坚持辩证的、螺旋上升的列宁主义政治报路线。这条路线强调政治报纸作为革命舆论机关和组织成员与群众之间的耦合器作用,通过“报纸——实践——报纸”即“从报纸到报纸”的良性循环,实现革命力量的不断壮大。它既反对机械融工对自发性的迷信,也同时反对把政治报变成脱离实践的孤立任务,而是要把政治报纸与有组织的群众实践紧密结合,形成从理论到群众、从群众到理论的反复飞跃:
|
||||
>
|
||||
> 首先通过政治报纸广泛聚拢先进工人、革命学生和进步知识分子,进行政治揭露和政治灌输,形成初步的理论共识和革命骨干。接着,把这些聚拢起来的人有计划地反向推动到有组织的地方革命实践当中去,然后在工厂、社区等阵地建立基层组织、开展政治教育和斗争锻炼,同时把实践经验和群众创造反哺到报纸中。进而,以扩大了的群众基础和丰富了的实践内容,进一步提升政治报纸的理论深度、报道广度和组织凝聚力,形成从低级到高级、从分散到集中的螺旋发展,最终服务于实质性建党。
|
||||
|
||||
 
|
||||
|
||||
> 我们坚持列宁同志关于从外部灌输社会主义意识的原理,同时强调灌输必须与有组织的群众实践相结合,不能使斗争工作变成少数人的清谈。我们主张通过政治报纸培养出称职的兼职革命家和群众组织,反对任何把自发斗争绝对化或把报纸作用绝对化、极端化的偏差。在当前阶段,重点是把政治报纸作为组织工具和连接线上、线下各部各组织的纽带,吸引先进分子,并把他们反向派往工厂实践,建立地下与地上相结合的基层组织,通过实践检验和丰富报纸内容,形成全国协调统筹的先锋队组织架构和力量,并最终在不断的线下实践斗争当中去促成真正的职业革命家和革命组织(革命党)的形成。
|
||||
>
|
||||
> 以组织为中心,建立起一套卫星式的报刊体系。分层建立起一套分属适用于各个阶段的刊物,例如《工人革命报》、《反修》、《文汇报》等,作为革命舆论机关和组织的耦合器。其内容必须紧密结合中国的无产阶级斗争实际,揭露修正主义政权的本质,传播革命的理论,批判机会主义路线。报纸要坚持独立性,在民主集中制的基础上团结革命力量,但绝不允许变成任何小集团的工具。当前的重点是提高理论质量、扩大发行网络、建立稳定供稿和发行队伍,为实质建党去搭建组织框架,并通过一切方式把先进分子组织起来,反向推动实践与战斗组织建设任务。把通过报纸聚拢的先进分子有计划地通过已经开始行动起来的地方串联小组,来组织起来反向派往工厂、工业区和社区,在有组织的状态下开展群众工作和斗争。不是散漫或自发,而是以革命理论为指导,融入、建立基层小组,培养出工人骨干,进行政治教育和组织锻炼。实践成果要及时反馈,形成“实践出真知、报纸出方向”的良性循环。重点工业区域要实现协调,避免各自为战。
|
||||
@@ -57,7 +54,7 @@
|
||||
而工革报之前联合自由派分子“九”搞出来的统战大纲也明确提到了等级制和经济刺激。
|
||||
|
||||
|
||||

|
||||

|
||||
|
||||
|
||||
重复一下我们之前的批判:
|
||||
|
||||
@@ -1,12 +1,6 @@
|
||||
# 燎原大群路线的实质是林彪主义
|
||||
**编者按:**
|
||||
|
||||
> 这篇文章材料充分,逻辑清晰的论证了大群的林彪主义实质。
|
||||
>
|
||||
>
|
||||
> 我们确定大群所谓的纪律和无产阶级的革命纪律毫无共同之处,其本质是一种庸俗的反毛主义的林彪主义。大群现在已经不是堕入机会主义路线的问题了,其内部被一些德才均不配位的小人和野心分子窃取了领导权,这些人借抽去了革命内容的纪律形式对内实行资产阶级专政,对外搞政治流氓行为,大群正在被这些贪图个人名利的可怜虫带上反革命的道路。
|
||||
>
|
||||
> 我们号召大群内部的同志造他们的反,打倒这些扮演拿破仑的波拿巴小丑。
|
||||
编者按:
|
||||
这篇文章材料充分,逻辑清晰的论证了大群的林彪主义实质。我们确定大群所谓的纪律和无产阶级的革命纪律毫无共同之处,其本质是一种庸俗的反毛主义的林彪主义。大群现在已经不是堕入机会主义路线的问题了,其内部被一些德才均不配位的小人和野心分子窃取了领导权,这些人借抽去了革命内容的纪律形式对内实行资产阶级专政,对外搞政治流氓行为,大群正在被这些贪图个人名利的可怜虫带上反革命的道路。我们号召大群内部的同志造他们的反,打倒这些扮演拿破仑的波拿巴小丑。
|
||||
|
||||
自23年远山高炬等发布黑路线以来,“马列毛主义与革命左翼大群”(燎原/布站)由高呼“到工人中去”的先进组织,一路堕落成一个不断阻碍同志融工、疯狂污蔑推进融工的组织是“无组织”的反革命大群。本文试论大群黑路线的实质,希望深化对大群修正主义本质的探讨,把真理越辩越明,以打击修正主义的根源。
|
||||
|
||||
|
||||
@@ -1,9 +1,9 @@
|
||||
# 无产阶级纪律必须在群众工作中建设——«左派幼稚病»揭露“野火”反动的纪律路线
|
||||
|
||||
野火说他们学了《共产主义运动中的“左派”幼稚病》(以下简称为《左派幼稚病》),就纪律问题对我们进行了回应,结果却好像根本没看过《左派幼稚病》一样。《左派幼稚病》中已经说明了纪律形成的条件:
|
||||
> 无产阶级革命政党的纪律是靠什么来维持的?是靠什么来检验的?是靠什么来加强的?第一,是靠无产阶级先锋队的觉悟和它对革命的忠诚,是靠它的坚韧不拔、自我牺牲和英雄气概。第二,是靠它善于同最广大的劳动群众,首先是同无产阶级劳动群众,但同样也同非无产阶级劳动群众联系、接近,甚至可以说在某种程度上同他们打成一片。第三,是靠这个先锋队所实行的政治领导正确,靠它的政治战略和策略正确,而最广大的群众根据切身经验也确信其正确。
|
||||
|
||||
>
|
||||
> 无产阶级革命政党的纪律是靠什么来维持的?是靠什么来检验的?是靠什么来加强的?第一,是靠无产阶级先锋队的觉悟和它对革命的忠诚,是靠它的坚韧不拔、自我牺牲和英雄气概。第二,是靠它善于同最广大的劳动群众,首先是同无产阶级劳动群众,但同样也同非无产阶级劳动群众联系、接近,甚至可以说在某种程度上同他们打成一片。第三,是靠这个先锋队所实行的政治领导正确,靠它的政治战略和策略正确,而最广大的群众根据切身经验也确信其正确。一个革命政党,要真正能够成为必将推翻资产阶级并改造整个社会的先进阶级的政党,没有上述条件,就不可能建立起纪律。没有这些条件,建立纪律的企图,就必然会成为空谈,成为漂亮话,成为装模作样。可是另一方面,这些条件又不能一下子就产生。只有经过长期的努力和艰苦的实践才能造成这些条件;正确的革命理论──而理论并不是教条──会使这些条件容易造成,但只有同真正群众性的和真正革命的运动的实践密切地联系起来,这些条件才能最终形成。
|
||||
> 一个革命政党,要真正能够成为必将推翻资产阶级并改造整个社会的先进阶级的政党,没有上述条件,就不可能建立起纪律。没有这些条件,建立纪律的企图,就必然会成为空谈,成为漂亮话,成为装模作样。可是另一方面,这些条件又不能一下子就产生。只有经过长期的努力和艰苦的实践才能造成这些条件;正确的革命理论──而理论并不是教条──会使这些条件容易造成,但只有同真正群众性的和真正革命的运动的实践密切地联系起来,这些条件才能最终形成。
|
||||
|
||||
列宁已经清楚地总结了无产阶级先锋队所需的纪律的三个形成条件。脱离这三点进行工作的话,是无法建立和维持革命政党的纪律的。革命路线的制定,不能不考虑这三个条件,也只有在正确的路线指导下,才能让这些条件在革命者的工作中逐渐形成并成熟。可以看见列宁的纪律路线完全是政治挂帅的。
|
||||
|
||||
@@ -21,12 +21,10 @@
|
||||
|
||||
取得对工人运动的领导权,这个之前已有广泛的经验,如“到工人中去”的经验,以及沈梦雨在广州日弘厂做的工作,这在我们长期扎根的情况下不难做到。而要能够在领导运动的过程中与资产阶级的路线进行斗争,与工人阶级中的自发性进行斗争,没有斗争前已经开展的地下工会中的思想路线工作,是做不到的。
|
||||
|
||||
>
|
||||
> 一个革命政党,要真正能够成为必将推翻资产阶级并改造整个社会的先进阶级的政党,没有上述条件,就不可能建立起纪律。没有这些条件,建立纪律的企图,就必然会成为空谈,成为漂亮话,成为装模作样。
|
||||
|
||||
列宁所列出的这几点,就是为了强调要同志们认识到“发挥与培养自觉性”、“与群众广泛联系”、“坚持政治挂帅”的重要性,而野火不正是这么装模作样地,脱离工人工作等方面来谈组织纪律吗?他们批量产出对厂内工作毫无帮助的水文的纪律,根本不是无产阶级组织的纪律,恰恰是对无产阶级的与群众联系这一纪律的最大破坏。
|
||||
|
||||
>
|
||||
> 可是另一方面,这些条件又不能一下子就产生。只有经过长期的努力和艰苦的实践才能造成这些条件;正确的革命理论──而理论并不是教条──会使这些条件容易造成,但只有同真正群众性的和真正革命的运动的实践密切地联系起来,这些条件才能最终形成。
|
||||
|
||||
我们说我们仍然处于手工业阶段,是因为工业化的组织和自觉的纪律,只有在真正的群众性的革命的运动实践中才能够形成。线上线下要共同工作起来,并不是一件容易的事,这需要融工同志去把线上组织改造成能够一起进行工人工作的组织。像列宁说的,这只有经过长期的努力和艰苦的实践才能实现。当然了,前提是在正确的革命理论,正确的革命路线的指导下的努力和实践。机会主义的路线,不管口号说的多么动听,实际则是在抵触工人群众,拒绝同工人群众的结合。在这种错误路线的指导下,缺乏了与群众性的紧密联系,这些条件难道能顺利形成吗?
|
||||
@@ -40,27 +38,18 @@
|
||||
关于线上工作的纪律性问题,主要的是关于线上工作者与厂内同志的结合的问题,首要的是厂内同志领导线上的问题,而厂内同志要能真正领导线上,是需要经过实践的探索的。正如工人阶级真正占领上层建筑需要通过艰苦的继续革命一样,厂内同志要学会领导线上同志为厂内工作服务,也需要艰苦的实践。
|
||||
|
||||
麒麟在文中提到了线上供稿中的一些积极性的问题,这主要地是供稿是否真正为工人工作服务的问题,供稿如果不为工人工作服务,写作的方向就会不明确,线上同志写作中就不会明确感到自己在走工人革命路线,那自然就难以积极地进行写作。麒麟的老前辈远山,早就就线上组织中的纪律问题提出:
|
||||
|
||||
>
|
||||
> 先扪心自问一下,机会主义者们内部是否存在以下这种情况:布置下去一件任务,但是最后没有人管理,也就没有人完成;或者是某天同志提了一个建议,但是当时有人进行回应,之后就没有人管,提出建议的本人也根本想不起来自己提出过这个建议。
|
||||
|
||||
远山所说的和麒麟所说的几乎是一模一样的。修正主义者往往通过提出一个正确的问题,然后提出修正主义的解法,以此刮起一股风来修正革命路线。为什么会出现远山和麒麟说的,一项建议就连提议的本人也根本想不起来呢?这是因为这个建议没有真正地切中厂内的需求。
|
||||
|
||||
真正的建成革命的纪律性的方法,正如上面我们论述的《左派幼稚病》中的路线,在于真正把线上工作和线下工作结合起来,让线上工作者在工作中真正认同线下的路线,真正感受到是在为线下的工作服务。线上线下的脱节是两方面的原因造成的,一方面是麒麟这样不愿意接受厂内同志领导的,这种相对较少的,更多的是因为厂内同志还没有学会领导线上工作,以及还没有很好地改造线上工作。《工人阶级必须领导一切》中说:
|
||||
|
||||
>
|
||||
> 工人阶级有阶级斗争、生产斗争、科学实验三大革命运动的丰富的实践经验,工人阶级最仇恨一切反社会主义、反毛泽东思想的反革命言行,最痛恨为剥削阶级服务的旧教育制度,最反对某些知识分子破坏国家财产、阻碍斗、批、改的“打内战”行动,最讨厌言行不一的空谈习气和两面派作风。所以,工人阶级群众同无产阶级专政的主要支拄——中国人民解放军的战士相结合,能够最有力地制止一切违反毛主席革命路线的错误倾向,最有效地解决种种所谓“老大难”的问题。知识分子纠缠不休不能解决的矛盾,工人们一到,很快就解决了。
|
||||
|
||||
融工同志是最直接地参与到厂内的三大实践当中的,他们是有最明确的工作需求的,所以融工同志真正需要的东西,那是不可能提出来又忘记的,因为这个需求会在厂内实践中反复反映出来。所以,一切线上和其他群体中的因为缺乏方向而停滞的工作,融工同志一旦发挥领导作用,很快就会有进展。所以,要反对形左实右的唯线下工作论,真正的融工工作应该是同时利用线上和线下的力量来推进厂内工作。正如《工人阶级必须领导一切》所说:
|
||||
|
||||
>
|
||||
> 工人宣传队要有步骤、有计划地到大、中、小学去,到上层建筑各个领域中去,到一切还没有搞好斗、批、改的单位去,…,为了完成这个历史任务,工人阶级必须认真学好毛泽东思想,学习毛主席历来教导的群众路线和调查研究的作风,不断提高自己的政治觉悟,加强革命的纪律性,不断批判工人阶级内部各种资产阶级腐朽作风的侵蚀及影响。
|
||||
|
||||
融工同志应该有步骤有计划地改造线上的一切工作,让线上工作服务于线下工作。在线下领导线上的过程中来让线上了解线下的需求,并且让线上得到工作成果的反馈,这样才能让线上了解线下,实现线下工作和线上工作的充分结合。线上如果没有线下的指导,就不可能为线下服务。工人如果不来领导,那么线上的同志就容易被机会主义分子,被其他阶级领导。
|
||||
|
||||
野火批判我们没有纪律,然而我们正是依靠同志们自觉的纪律迅速揭露了麒麟这一林彪主义者。在同志们自觉维护的无产阶级专政的震慑下,麒麟连自己批判我们的文章都不敢公开在我们群上发表,还是我们在他叛逃后帮他发布的。在厂内的领导下,同志们继承批林批孔运动的精神,就这位反面教员开展了生动活泼的集体批判、研究和写作。
|
||||
|
||||
我们相信野火中仍然有受蒙蔽的要革命的同志,相信他们学习了《左派幼稚病》后,会意识到脱离厂内工作来建立革命纪律是何等幼稚、野火吹嘘量产对厂内工作毫无用处的水文是多么可笑。要革命的同志们终究会和工农解放社的同志们一起,沿着列宁指引的道路,投身融工与服务融工工作的浪潮中。还是那句老话,厂内广阔天地,大有可为!
|
||||
|
||||
|
||||
|
||||
我们相信野火中仍然有受蒙蔽的要革命的同志,相信他们学习了《左派幼稚病》后,会意识到脱离厂内工作来建立革命纪律是何等幼稚、野火吹嘘量产对厂内工作毫无用处的水文是多么可笑。要革命的同志们终究会和工农解放社的同志们一起,沿着列宁指引的道路,投身融工与服务融工工作的浪潮中。还是那句老话,厂内广阔天地,大有可为!
|
||||
@@ -18,8 +18,6 @@
|
||||
### 文章要点
|
||||
文中记录了作者在厂内工作培养一位工人同志的过程。这位工人同志最初幻想当保安改善自身处境,后遇到问题时,作者及时结合具体困难进行分析,纠正了工人同志的错误倾向,通过持续帮助和灌输,工人同志逐渐认清了资本主义本质,开始主动进厂并引导身边受压迫学生。
|
||||
|
||||
这位工人同志还与一个线上联系工人只讲理论的偏政治报路线的读书小组进行了交流,他感觉聊不来因为讨论哲学在厂内实际工作中作用不大,这反映了政治报和融工派两条路线的差距。
|
||||
|
||||
文中案例展示了发展工人同志的有效方法,应围绕工人当下生活矛盾展开工作,在提供实际帮助的同时,持续进行思想教育和斗争,帮助发展对象在路线上达成一致。
|
||||
|
||||
### 案例对我做学生群众工作的启发
|
||||
|
||||
@@ -5,19 +5,19 @@
|
||||
12 月 27 日,长沙市某省级示范高中爆发了学生抗议活动。据悉,因校方将原定的三天元旦假期突然缩减为两天,引发了学生及部分老师的不满。当天,学生们开始在食堂、年级办公室张贴并传阅抗议檄文和大字报,并进而演变成自发聚集喊楼行动,喊出“誓死力争,还我三休”等口号。最终学生们抗议成功,校方恢复了三天假期。
|
||||
|
||||
**这是学生们打出的大字报**:
|
||||

|
||||

|
||||
|
||||
|
||||

|
||||

|
||||
|
||||
**据传,学校的官僚们也曾试图处理学生**:
|
||||
|
||||

|
||||

|
||||
|
||||
但不管结局如何,学生聚集起来的力量最终还是让官僚们退步了。因为学生们的团结他们也
|
||||
不敢处理学生,并且最终承认了学生们的合法三休权益。学生们事后总结了这场斗争:
|
||||
|
||||

|
||||

|
||||
|
||||
学生们意识到了团结和斗争的作用,他们用大字报等形式把学生的愤怒集中的表现了出来并凝聚为共同的行动。因此,这里的大字报完全起到了集中表述阶级情绪、阶级利益和集中阶级行动作用。同时,大字报等行为和运动本身也推动了整个学生的进步,使原本茫然服从学校秩序和服务官僚资产阶级利益的学生们开始明确的把自身和学校官僚们对立了起来,在自身集体利益和学校官僚利益之间选择了前者。
|
||||
|
||||
@@ -41,4 +41,4 @@
|
||||
>
|
||||
> -- 革命社会主义阵地(RSF)第八期月刊《是斗争的胜利!论伟大的“九十学运”的意义》
|
||||
|
||||
“九十学运”中所揭示和所预见的全部学生运动发展趋势在于:提高现有的学生运动的数量和规模,最终要把它发展到包含全国学生的广泛维护自身合法利益的程度;提高学生斗争的组织程度,像郑州一中学生所一开始要求的那样,要建立起包含最广泛的学生的组织,这种组织是主要用来进行阶级斗争的;在运动中逐步提高学生的政治性,使它开始和社会主义运动结合起来,这就要建立领导整个学生运动的学生政治组织。因此,长沙此校学生的斗争意义就在于证实了这些趋势和预见是完全正确的和完全可行的,我们今后的学生工作应该照着这样的方向稳步的前进。
|
||||
“九十学运”中所揭示和所预见的全部学生运动发展趋势在于:提高现有的学生运动的数量和规模,最终要把它发展到包含全国学生的广泛维护自身合法利益的程度;提高学生斗争的组织程度,像郑州一中学生所一开始要求的那样,要建立起包含最广泛的学生的组织,这种组织是主要用来进行阶级斗争的;在运动中逐步提高学生的政治性,使它开始和社会主义运动结合起来,这就要建立领导整个学生运动的学生政治组织。因此,长沙此校学生的斗争意义就在于证实了这些趋势和预见是完全正确的和完全可行的,我们今后的学生工作应该照着这样的方向稳步的前进。
|
||||
@@ -8,7 +8,7 @@ OpenPGP来自于PGP加密程序,是一种公有领域的数据检验和加密
|
||||
|
||||
以上加密技术可以通过多种方法实现。本文中,我们详细介绍Proton邮箱内置的PGP端到端加密技术。
|
||||
|
||||
Proton邮箱的官网是proton.me,使用者应当按照第四期的注意事项注册并管理该邮箱。当创建一个账户时,邮箱会生成两个公私钥对,一个验证登陆,一个加密邮件。可以在邮箱内的Settings → All settings → Proton Mail → Encryption and keys选项中进行管理:
|
||||
Proton邮箱的官网是 proton.me ,使用者应当按照第四期的注意事项注册并管理该邮箱。当创建一个账户时,邮箱会生成两个公私钥对,一个验证登陆,一个加密邮件。可以在邮箱内的Settings → All settings → Proton Mail → Encryption and keys选项中进行管理:
|
||||
|
||||
如果双方均使用Proton邮箱,则整个过程不需要任何额外的操作,邮件的内容均自动使用pgp加密并签名,获取公钥的过程将自动完成。然而在实际情况下,我们有时需要同墙内的某人进行交流,而不希望被网警获取内容。Proton确实提供了常规密码保护邮件的功能,但是接收方的密码验证必须通过被墙的Proton服务器进行。所以最好是仍然采用PGP加密的形式,而这要求双方持有对方的公钥。由于公钥服务器也可能被墙,所以可以直接用邮件向墙内邮箱发出自己的公钥。以下步骤要求对方的邮箱或插件支持PGP。
|
||||
|
||||
@@ -18,7 +18,7 @@ Proton邮箱的官网是proton.me,使用者应当按照第四期的注意事
|
||||
|
||||
如图,可以非常直观方便地完成密钥生成,加密并签名邮件,解密并验证邮件的步骤。
|
||||
|
||||
然而,这种第三方在线工具都不能避免安全性的问题,在墙内更是如此。所以,我们比较推荐使用著名的GnuPG工具——这是自由软件基金会对于OpenPGP标准的自由软件实现。该工具的使用教程在emailselfdefense.fsf.org/zh-hans/网页。这个工具原生支持linux,但在Windows和macOS上也能使用。需要说明的是,和教程不同,我们这里是将其当作纯粹的离线的加解密和密钥生成工具使用,因此全过程应当离线且不上传密钥,也不要将其嵌入任何不可靠邮箱作为自动插件使用(即使有此选项)。因此这个过程不会出现图形界面,需要在命令行内完成,命令手册在https://www.gnupg.org/gph/en/manual/x110.html,或者可在命令行内键入gpg–help查看。
|
||||
然而,这种第三方在线工具都不能避免安全性的问题,在墙内更是如此。所以,我们比较推荐使用著名的GnuPG工具——这是自由软件基金会对于OpenPGP标准的自由软件实现。该工具的使用教程在 emailselfdefense.fsf.org/zh-hans/ 网页。这个工具原生支持linux,但在Windows和macOS上也能使用。需要说明的是,和教程不同,我们这里是将其当作纯粹的离线的加解密和密钥生成工具使用,因此全过程应当离线且不上传密钥,也不要将其嵌入任何不可靠邮箱作为自动插件使用(即使有此选项)。因此这个过程不会出现图形界面,需要在命令行内完成,命令手册在 https://www.gnupg.org/gph/en/manual/x110.html ,或者可在命令行内键入gpg–help查看。
|
||||
|
||||
过程完成以后,对方应当也已经生成了自己的公私钥了。然后他也可以将他的公钥发回给你
|
||||
|
||||
|
||||
@@ -33,6 +33,8 @@
|
||||
}
|
||||
|
||||
.global h1 {
|
||||
font-size: 3rem;
|
||||
|
||||
line-height: 1.5;
|
||||
}
|
||||
.global h2 {
|
||||
@@ -61,18 +63,39 @@
|
||||
color: white;
|
||||
background-color: red;
|
||||
border: 0.18rem solid red;
|
||||
display: inline-flex;
|
||||
display: block;
|
||||
break-before: page;
|
||||
page-break-before: always;
|
||||
width: fit-content;
|
||||
line-height: 1.5;
|
||||
padding: 0.08em 0.24em;
|
||||
white-space: normal;
|
||||
word-break: break-all;
|
||||
font-family: "ZCOOL QingKe HuangYou", sans-serif;
|
||||
margin: 1rem 1rem 0rem 2rem;
|
||||
margin: 0 1rem 1.5rem 2rem;
|
||||
border-radius: 0.5rem;
|
||||
overflow-wrap: break-word;
|
||||
max-width: 21cm;
|
||||
}
|
||||
|
||||
/* Keep the first article with its column title; separate later articles
|
||||
while allowing their bodies to flow across pages. */
|
||||
.passages {
|
||||
margin: 0 2rem;
|
||||
padding: 0 0 1.5rem;
|
||||
break-inside: auto;
|
||||
page-break-inside: auto;
|
||||
}
|
||||
|
||||
.passages:not(.first-passage) {
|
||||
break-before: page;
|
||||
page-break-before: always;
|
||||
}
|
||||
|
||||
.passages.first-passage > :first-child {
|
||||
margin-block-start: 1.5rem;
|
||||
}
|
||||
|
||||
.index-column page_title {
|
||||
font-size: calc(var(--font-size) + 0.5rem);
|
||||
font-weight: bold;
|
||||
@@ -85,14 +108,95 @@
|
||||
margin: 1rem 0;
|
||||
color: #666;
|
||||
}
|
||||
|
||||
/* Material You-inspired surfaces for Markdown-generated tables. */
|
||||
.global table {
|
||||
width: 100%;
|
||||
margin: 1.5rem 0 2rem;
|
||||
border: 1px solid #c4c7c5;
|
||||
border-radius: 0.75rem;
|
||||
border-spacing: 0;
|
||||
border-collapse: separate;
|
||||
background-color: #fdfbff;
|
||||
color: #1a1b1f;
|
||||
overflow: hidden;
|
||||
break-inside: auto;
|
||||
page-break-inside: auto;
|
||||
line-height: 1.5;
|
||||
}
|
||||
|
||||
.global table caption {
|
||||
margin-bottom: 0.75rem;
|
||||
color: #454746;
|
||||
font-size: 0.9em;
|
||||
font-weight: 600;
|
||||
text-align: left;
|
||||
}
|
||||
|
||||
.global table thead {
|
||||
display: table-header-group;
|
||||
background-color: #dcefeb;
|
||||
color: #00201f;
|
||||
}
|
||||
|
||||
.global table th,
|
||||
.global table td {
|
||||
padding: 0.75rem 1rem;
|
||||
border-right: 1px solid #c4c7c5;
|
||||
border-bottom: 1px solid #c4c7c5;
|
||||
text-align: left;
|
||||
vertical-align: top;
|
||||
overflow-wrap: anywhere;
|
||||
}
|
||||
|
||||
.global table th {
|
||||
color: #00504f;
|
||||
font-weight: 700;
|
||||
}
|
||||
|
||||
.global table tbody tr:nth-child(even) {
|
||||
background-color: #f3f5f3;
|
||||
}
|
||||
|
||||
.global table tbody tr {
|
||||
break-inside: avoid;
|
||||
page-break-inside: avoid;
|
||||
}
|
||||
|
||||
.global table tr > :last-child {
|
||||
border-right: 0;
|
||||
}
|
||||
|
||||
.global table tbody tr:last-child td {
|
||||
border-bottom: 0;
|
||||
}
|
||||
|
||||
.global table thead tr:first-child th:first-child {
|
||||
border-top-left-radius: 0.75rem;
|
||||
}
|
||||
|
||||
.global table thead tr:first-child th:last-child {
|
||||
border-top-right-radius: 0.75rem;
|
||||
}
|
||||
|
||||
.global table tbody tr:last-child td:first-child {
|
||||
border-bottom-left-radius: 0.75rem;
|
||||
}
|
||||
|
||||
.global table tbody tr:last-child td:last-child {
|
||||
border-bottom-right-radius: 0.75rem;
|
||||
}
|
||||
|
||||
.global img {
|
||||
height: 100%;
|
||||
object-fit: scale-down;
|
||||
height: auto;
|
||||
max-width: 100%;
|
||||
max-height: 25cm;
|
||||
border-radius: 12px;
|
||||
border: 4px solid linear-gradient(45deg, #e799b1, #44b144, #2f90ba);
|
||||
box-shadow: 0 10px 20px rgba(0, 0, 0, 0.1);
|
||||
max-height: 33vh;
|
||||
object-fit: contain;
|
||||
padding: 5px;
|
||||
border: 1px solid #c9d9d5;
|
||||
border-radius: 10px;
|
||||
background: #fff;
|
||||
box-shadow: 0 2px 8px rgba(24, 50, 46, 0.14);
|
||||
display: block;
|
||||
margin: 1rem auto;
|
||||
/* color: transparent; */
|
||||
|
||||
@@ -49,11 +49,23 @@
|
||||
/* Use padding instead of adjacent margins so the article separators remain
|
||||
visible when Chromium fragments the document for printing. */
|
||||
.passages {
|
||||
padding-block: 2.5rem;
|
||||
margin: 0 2rem;
|
||||
padding: 0 0 2.5rem;
|
||||
break-inside: auto;
|
||||
page-break-inside: auto;
|
||||
}
|
||||
|
||||
/* Keep the column title with its first article, but start later articles
|
||||
on a fresh page while allowing each article body to fragment naturally. */
|
||||
.passages:not(.first-passage) {
|
||||
break-before: page;
|
||||
page-break-before: always;
|
||||
}
|
||||
|
||||
.passages.first-passage > :first-child {
|
||||
margin-block-start: 1.5rem;
|
||||
}
|
||||
|
||||
.global h1 {
|
||||
line-height: 4rem;
|
||||
font-family: "HongLei", sans-serif;
|
||||
@@ -84,14 +96,16 @@
|
||||
color: white;
|
||||
background-color: red;
|
||||
border: 0.18rem solid red;
|
||||
display: inline-flex;
|
||||
display: block;
|
||||
break-before: page;
|
||||
page-break-before: always;
|
||||
width: fit-content;
|
||||
line-height: 1.5;
|
||||
padding: 0.08em 0.24em;
|
||||
white-space: normal;
|
||||
word-break: break-all;
|
||||
font-family: "ZCOOL QingKe HuangYou", sans-serif;
|
||||
margin: 4rem 1rem 1rem 2rem;
|
||||
margin: 0 1rem 1.5rem 2rem;
|
||||
border-radius: 0.5rem;
|
||||
overflow-wrap: break-word;
|
||||
max-width: 21cm;
|
||||
@@ -103,13 +117,100 @@
|
||||
padding: 0.5rem 1rem;
|
||||
margin: 1rem 0;
|
||||
color: #666;
|
||||
break-inside: auto;
|
||||
page-break-inside: auto;
|
||||
break-after: auto;
|
||||
page-break-after: auto;
|
||||
}
|
||||
|
||||
/* Material You-inspired table surfaces for Markdown-generated tables. */
|
||||
.global table {
|
||||
width: 100%;
|
||||
margin: 1.5rem 0 2rem;
|
||||
border: 1px solid #c4c7c5;
|
||||
border-radius: 0.75rem;
|
||||
border-spacing: 0;
|
||||
border-collapse: separate;
|
||||
background-color: #fdfbff;
|
||||
color: #1a1b1f;
|
||||
overflow: hidden;
|
||||
break-inside: auto;
|
||||
page-break-inside: auto;
|
||||
line-height: 1.5;
|
||||
}
|
||||
|
||||
.global table caption {
|
||||
margin-bottom: 0.75rem;
|
||||
color: #454746;
|
||||
font-size: 0.9em;
|
||||
font-weight: 600;
|
||||
text-align: left;
|
||||
}
|
||||
|
||||
.global table thead {
|
||||
display: table-header-group;
|
||||
background-color: #dcefeb;
|
||||
color: #00201f;
|
||||
}
|
||||
|
||||
.global table th,
|
||||
.global table td {
|
||||
padding: 0.75rem 1rem;
|
||||
border-right: 1px solid #c4c7c5;
|
||||
border-bottom: 1px solid #c4c7c5;
|
||||
text-align: left;
|
||||
vertical-align: top;
|
||||
overflow-wrap: anywhere;
|
||||
}
|
||||
|
||||
.global table th {
|
||||
color: #00504f;
|
||||
font-weight: 700;
|
||||
}
|
||||
|
||||
.global table tbody tr:nth-child(even) {
|
||||
background-color: #f3f5f3;
|
||||
}
|
||||
|
||||
.global table tbody tr {
|
||||
break-inside: avoid;
|
||||
page-break-inside: avoid;
|
||||
}
|
||||
|
||||
.global table tr > :last-child {
|
||||
border-right: 0;
|
||||
}
|
||||
|
||||
.global table tbody tr:last-child td {
|
||||
border-bottom: 0;
|
||||
}
|
||||
|
||||
.global table thead tr:first-child th:first-child {
|
||||
border-top-left-radius: 0.75rem;
|
||||
}
|
||||
|
||||
.global table thead tr:first-child th:last-child {
|
||||
border-top-right-radius: 0.75rem;
|
||||
}
|
||||
|
||||
.global table tbody tr:last-child td:first-child {
|
||||
border-bottom-left-radius: 0.75rem;
|
||||
}
|
||||
|
||||
.global table tbody tr:last-child td:last-child {
|
||||
border-bottom-right-radius: 0.75rem;
|
||||
}
|
||||
|
||||
.global img {
|
||||
max-width: 100%;
|
||||
height: auto;
|
||||
border-radius: 12px;
|
||||
border: 4px solid linear-gradient(45deg, #e799b1, #44b144, #2f90ba);
|
||||
box-shadow: 0 10px 20px rgba(0, 0, 0, 0.1);
|
||||
max-height: 33vh;
|
||||
object-fit: contain;
|
||||
padding: 5px;
|
||||
border: 1px solid #c9d9d5;
|
||||
border-radius: 10px;
|
||||
background: #fff;
|
||||
box-shadow: 0 2px 8px rgba(24, 50, 46, 0.14);
|
||||
display: block;
|
||||
margin: 1rem auto;
|
||||
/* color: transparent; */
|
||||
|
||||
@@ -93,6 +93,85 @@
|
||||
border-radius: 22px;
|
||||
border-color: #666;
|
||||
}
|
||||
|
||||
/* Material You-inspired surfaces for Markdown-generated tables. */
|
||||
.global table {
|
||||
width: 100%;
|
||||
margin: 1.5rem 0 2rem;
|
||||
border: 1px solid #c4c7c5;
|
||||
border-radius: 0.75rem;
|
||||
border-spacing: 0;
|
||||
border-collapse: separate;
|
||||
background-color: #fdfbff;
|
||||
color: #1a1b1f;
|
||||
overflow: hidden;
|
||||
break-inside: auto;
|
||||
page-break-inside: auto;
|
||||
line-height: 1.5;
|
||||
}
|
||||
|
||||
.global table caption {
|
||||
margin-bottom: 0.75rem;
|
||||
color: #454746;
|
||||
font-size: 0.9em;
|
||||
font-weight: 600;
|
||||
text-align: left;
|
||||
}
|
||||
|
||||
.global table thead {
|
||||
display: table-header-group;
|
||||
background-color: #dcefeb;
|
||||
color: #00201f;
|
||||
}
|
||||
|
||||
.global table th,
|
||||
.global table td {
|
||||
padding: 0.75rem 1rem;
|
||||
border-right: 1px solid #c4c7c5;
|
||||
border-bottom: 1px solid #c4c7c5;
|
||||
text-align: left;
|
||||
vertical-align: top;
|
||||
overflow-wrap: anywhere;
|
||||
}
|
||||
|
||||
.global table th {
|
||||
color: #00504f;
|
||||
font-weight: 700;
|
||||
}
|
||||
|
||||
.global table tbody tr:nth-child(even) {
|
||||
background-color: #f3f5f3;
|
||||
}
|
||||
|
||||
.global table tbody tr {
|
||||
break-inside: avoid;
|
||||
page-break-inside: avoid;
|
||||
}
|
||||
|
||||
.global table tr > :last-child {
|
||||
border-right: 0;
|
||||
}
|
||||
|
||||
.global table tbody tr:last-child td {
|
||||
border-bottom: 0;
|
||||
}
|
||||
|
||||
.global table thead tr:first-child th:first-child {
|
||||
border-top-left-radius: 0.75rem;
|
||||
}
|
||||
|
||||
.global table thead tr:first-child th:last-child {
|
||||
border-top-right-radius: 0.75rem;
|
||||
}
|
||||
|
||||
.global table tbody tr:last-child td:first-child {
|
||||
border-bottom-left-radius: 0.75rem;
|
||||
}
|
||||
|
||||
.global table tbody tr:last-child td:last-child {
|
||||
border-bottom-right-radius: 0.75rem;
|
||||
}
|
||||
|
||||
.global img {
|
||||
height: auto;
|
||||
|
||||
|
||||
@@ -4,6 +4,64 @@ GP 是一个基于 Deno、Markdown 和 HTML 模板的刊物构建工具,可以
|
||||
PDF,也可以导出合并后的 Markdown 或纯文本。PDF 支持自动生成目录、PDF 线性化和
|
||||
OpenPGP 签名。
|
||||
|
||||
## 代码目录
|
||||
|
||||
开发文档:
|
||||
|
||||
- [文档输入输出接口开发指南](docs/document-api.md):HTML 中间表示、接口契约与扩展示例。
|
||||
- [build.js 开发指南](docs/build-js.md):执行时机、注入对象、DOM 修改和配置传播边界。
|
||||
- [转换流程 draw.io 源图](docs/diagrams/document-workflow.drawio)
|
||||
- [build.js 流程 draw.io 源图](docs/diagrams/build-js-workflow.drawio)
|
||||
|
||||
文档转换统一使用 HTML 中间表示,公共接口从 `document/mod.ts` 导出:
|
||||
|
||||
```text
|
||||
DocumentInput.read() → HtmlDocument → DocumentOutput.write() → OutputResult
|
||||
```
|
||||
|
||||
`PublicationInput` 读取出版物并渲染 Markdown 和模板变量;`HtmlInput` 接收已有的
|
||||
`HtmlDocument`,无需读取 Markdown 文件。中间文档包含出版物元数据、按顺序排列的
|
||||
HTML 章节(ID、标题、专栏、来源路径)、资源基址,以及封面、样式和前后附加内容。
|
||||
章节 HTML 保留原始资源引用,不含 PDF 定位标记。
|
||||
|
||||
`TextOutput` 支持 HTML、Markdown、TXT;`PdfOutput`
|
||||
负责浏览器排版、目录和后处理; `EpubOutput`
|
||||
负责章节链接和资源打包。图片嵌入和格式专用标记在输出端处理。 现有 Builder
|
||||
方法保留作兼容入口,新代码建议使用以下接口:
|
||||
|
||||
```ts
|
||||
import {
|
||||
convertDocument,
|
||||
PublicationInput,
|
||||
TextOutput,
|
||||
} from "./document/mod.ts";
|
||||
|
||||
const input = new PublicationInput(config, variables);
|
||||
const result = await convertDocument(input, new TextOutput(config, "html"));
|
||||
console.log(result.path, result.mediaType);
|
||||
```
|
||||
|
||||
同一份中间文档可交给多个输出端。新增输入实现
|
||||
`DocumentInput.read()`,新增格式实现
|
||||
`DocumentOutput.write()`,无需修改其他格式。输出接口不修改传入的中间文档。
|
||||
HTML、TXT 可分别配置 `html.path`、`txt.path`;未配置时从 `md.path` 派生。
|
||||
|
||||
`main.ts` 保留为命令入口,配置和输出路径仍相对于 `tools/gp` 工作目录解析。
|
||||
各模块的测试与实现放在同一目录:
|
||||
|
||||
```text
|
||||
tools/gp/
|
||||
├── main.ts # 命令入口和配置加载
|
||||
├── cli/ # 子命令分发与导出处理
|
||||
├── publication/ # 出版物、专栏、文章读取及排序
|
||||
├── template/ # 模板变量和模板渲染
|
||||
├── mdit/ # Markdown 解析及插件
|
||||
├── assets/ # 外部素材缓存
|
||||
├── pdf_builder/ # 文档渲染、PDF 目录、流水线及 Worker
|
||||
├── epub_builder/ # EPUB 章节和资源打包
|
||||
└── sync/ # WebDAV 上传、同步及清理
|
||||
```
|
||||
|
||||
## 工作流程
|
||||
|
||||
```text
|
||||
@@ -94,16 +152,22 @@ deno run -A main.ts --from 26.0.0 generate md
|
||||
# 将每个配置的文章合并导出为 Markdown
|
||||
deno run -A main.ts generate md
|
||||
|
||||
# 导出为 HTML(图片直接内嵌,路径从 md.path 派生)
|
||||
deno run -A main.ts generate html
|
||||
|
||||
# 根据 md.path 导出为纯文本(输出文件名中的 md 会替换为 txt)
|
||||
deno run -A main.ts generate txt
|
||||
|
||||
# 按配置中的 publication 生成 EPUB 3,包含可点击目录
|
||||
# 按配置中的 publication 生成 EPUB 3,包含可点击目录和 EPUB 包内图片
|
||||
deno run -A main.ts generate epub
|
||||
```
|
||||
|
||||
Markdown 和 HTML 导出会将本地图片嵌入为 Base64 data URL。EPUB 不使用 data URL,
|
||||
而是将图片作为 EPUB 包内资源保存并从章节引用,兼容 EPUB 阅读器的资源模型。
|
||||
|
||||
EPUB 默认使用 `md.path` 将扩展名替换为 `.epub`;也可在每个配置中设置
|
||||
`[settings.epub]` 的 `path`、`title`、`author` 和 `language`。每篇文章会拆成独立章节,
|
||||
EPUB 导航目录按出版物排序链接到各章节。
|
||||
`[settings.epub]` 的 `path`、`title`、`author` 和
|
||||
`language`。每篇文章会拆成独立章节, EPUB 导航目录按出版物排序链接到各章节。
|
||||
|
||||
### 本地素材缓存
|
||||
|
||||
@@ -199,7 +263,13 @@ draft = false
|
||||
地址,默认值为 `../../assets`。例如 Markdown 中的 ``
|
||||
会从该目录读取,不需要单独启动附件文件服务器。
|
||||
|
||||
目录搜索策略可以在模板的 `build.js` 中通过 `cfg.indexSearch` 调整。默认是
|
||||
PDF 目录优先读取文章标题或首段上的唯一链接标记
|
||||
`https://wpep.invalid/article-start-编号`,直接取得起始页码;标记齐全时不会提取或搜索正文。
|
||||
合并目录时会删除这些临时链接注释及其 URL,保留标题文字和文章中的普通链接,
|
||||
并将目录链接、PDF 书签指向插入目录后的对应页面。
|
||||
|
||||
只有缺少标记的文章才回退到标题全文搜索(例如处理旧
|
||||
PDF)。回退搜索策略可以在模板的 `build.js` 中通过 `cfg.indexSearch` 调整。默认是
|
||||
`single_jump`:逐页扫描,每页只读取一次页面文本,并匹配该页上所有尚未匹配的文章;未匹配到标题的页面不会中断后续查找:
|
||||
|
||||
```js
|
||||
@@ -376,7 +446,7 @@ deno test -A
|
||||
deno task dev
|
||||
|
||||
# 检查类型和格式
|
||||
deno check main.ts pdf_pipeline.ts
|
||||
deno check main.ts pdf_builder/pipeline.ts
|
||||
deno fmt --check
|
||||
```
|
||||
|
||||
|
||||
@@ -1,10 +1,15 @@
|
||||
//import TurndownService from "turndown";
|
||||
import { args, cfg } from "../main.ts";
|
||||
import { OpenPGPSigner } from "../pdf_builder/utils.ts";
|
||||
import { DocuBuilder } from "../pdf_builder/docu_builder.ts";
|
||||
import { createTemplateVariables } from "../template_engine.ts";
|
||||
import { prepareLocalCache } from "../local_cache.ts";
|
||||
import { EpubBuilder } from "../epub_builder.ts";
|
||||
import { PublicationInput } from "../document/inputs.ts";
|
||||
import { convertDocument } from "../document/types.ts";
|
||||
import {
|
||||
EpubOutput,
|
||||
type TextFormat,
|
||||
TextOutput,
|
||||
} from "../document/outputs.ts";
|
||||
import { createTemplateVariables } from "../template/engine.ts";
|
||||
import { prepareLocalCache } from "../assets/local_cache.ts";
|
||||
|
||||
function selectedConfigs() {
|
||||
return cfg.settings.filter((config: Record<string, unknown>) =>
|
||||
@@ -65,7 +70,7 @@ function runPdfWorker(
|
||||
return createTemplateVariables(args, config, cfg.vars).then((variables) =>
|
||||
new Promise<void>((resolve, reject) => {
|
||||
const worker = new Worker(
|
||||
new URL("../pdf_worker.ts", import.meta.url).href,
|
||||
new URL("../pdf_builder/worker.ts", import.meta.url).href,
|
||||
{
|
||||
type: "module",
|
||||
},
|
||||
@@ -109,64 +114,38 @@ function runPdfWorker(
|
||||
}
|
||||
|
||||
export async function generate_md() {
|
||||
await cacheBeforeGeneration();
|
||||
try {
|
||||
for (const config of selectedConfigs()) {
|
||||
const builder = new DocuBuilder(
|
||||
config,
|
||||
await createTemplateVariables(args, config, cfg.vars),
|
||||
);
|
||||
await generateText("md");
|
||||
}
|
||||
|
||||
console.debug(`Processing config: ${JSON.stringify(config)}`);
|
||||
builder.with_files_sync(config.journal.from_directory);
|
||||
await builder.as_markdown();
|
||||
const markdownPath = config.md?.path ?? config.markdown_output;
|
||||
if (!markdownPath) {
|
||||
throw new Error("Missing markdown output path in config. Use md.path.");
|
||||
}
|
||||
await builder.build_as_raw(markdownPath);
|
||||
}
|
||||
} finally {
|
||||
// No browser to close in this function
|
||||
}
|
||||
export async function generate_html() {
|
||||
await generateText("html");
|
||||
}
|
||||
|
||||
export async function generate_txt() {
|
||||
await generateText("txt");
|
||||
}
|
||||
|
||||
async function generateText(format: TextFormat) {
|
||||
await cacheBeforeGeneration();
|
||||
try {
|
||||
for (const config of selectedConfigs()) {
|
||||
const builder = new DocuBuilder(
|
||||
config,
|
||||
await createTemplateVariables(args, config, cfg.vars),
|
||||
);
|
||||
|
||||
console.debug(`Processing config: ${JSON.stringify(config)}`);
|
||||
builder.with_files_sync(config.journal.from_directory);
|
||||
await builder.as_txt();
|
||||
const markdownPath = config.md?.path ?? config.markdown_output;
|
||||
if (!markdownPath) {
|
||||
throw new Error("Missing txt output path in config. Use md.path.");
|
||||
}
|
||||
const txtPath = markdownPath.replace("md", "txt");
|
||||
|
||||
await builder.build_as_raw(txtPath);
|
||||
}
|
||||
} finally {
|
||||
// No browser to close in this function
|
||||
for (const config of selectedConfigs()) {
|
||||
const input = new PublicationInput(
|
||||
config,
|
||||
await createTemplateVariables(args, config, cfg.vars),
|
||||
);
|
||||
const result = await convertDocument(input, new TextOutput(config, format));
|
||||
console.log(`Generated ${format}: ${result.path}`);
|
||||
}
|
||||
}
|
||||
|
||||
export async function generate_epub() {
|
||||
await cacheBeforeGeneration();
|
||||
for (const config of selectedConfigs()) {
|
||||
const builder = new EpubBuilder(
|
||||
const input = new PublicationInput(
|
||||
config,
|
||||
await createTemplateVariables(args, config, cfg.vars),
|
||||
);
|
||||
builder.with_files_sync(config.journal.from_directory);
|
||||
const output = builder.outputPath();
|
||||
await builder.build(output);
|
||||
console.log(`Generated EPUB: ${output}`);
|
||||
const result = await convertDocument(input, new EpubOutput(config));
|
||||
console.log(`Generated EPUB: ${result.path}`);
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1,11 +1,16 @@
|
||||
import {
|
||||
generate_epub,
|
||||
generate_html,
|
||||
generate_md,
|
||||
generate_pdf,
|
||||
generate_txt,
|
||||
just_local_cache,
|
||||
} from "./handlers.ts";
|
||||
import { clean_local_files, sync_webdav, upload_webdav } from "../webdav.ts";
|
||||
import {
|
||||
clean_local_files,
|
||||
sync_webdav,
|
||||
upload_webdav,
|
||||
} from "../sync/webdav.ts";
|
||||
|
||||
export async function match_sub_command_to_function(
|
||||
sub_command: Array<string>,
|
||||
@@ -35,6 +40,9 @@ export async function match_sub_command_to_function(
|
||||
case "md":
|
||||
await generate_md();
|
||||
break;
|
||||
case "html":
|
||||
await generate_html();
|
||||
break;
|
||||
case "txt":
|
||||
await generate_txt();
|
||||
break;
|
||||
|
||||
@@ -0,0 +1,120 @@
|
||||
# build.js 开发指南
|
||||
|
||||
build.js 是完整 HTML 组装后的同步处理钩子,用于调整 DOM、样式或当前构建配置。
|
||||
它由 `DocuBuilder.as_html_page()` 执行,具体实现见
|
||||
[`pdf_builder/docu_builder.ts`](../pdf_builder/docu_builder.ts)。
|
||||
|
||||

|
||||
|
||||
[编辑 draw.io 源图](diagrams/build-js-workflow.drawio)。
|
||||
|
||||
## 配置与执行时机
|
||||
|
||||
```toml
|
||||
[settings.journal]
|
||||
build_js_script = "../../templates/computer_template/build.js"
|
||||
```
|
||||
|
||||
路径相对于进程工作目录解析,通常为 `tools/gp`,不相对于模板目录。
|
||||
禁用时应显式配置:
|
||||
|
||||
```toml
|
||||
build_js_script = false
|
||||
```
|
||||
|
||||
当前判断是 `build_js_script != false`;省略字段不会自动禁用,而会尝试读取
|
||||
`undefined` 路径,因此不要省略。脚本在章节模板和 Markdown 已渲染、HTML
|
||||
布局已组装、 本地章节图片已处理之后执行;PDF 的临时定位链接也已加入。
|
||||
同一中间文档被多次导出时,每次 HTML 组装都会执行一次脚本。
|
||||
|
||||
| 格式 | 是否执行 | 后续步骤 |
|
||||
| -------- | -------- | ---------------------------------------- |
|
||||
| PDF | 是 | 浏览器分页、目录定位及合并、书签和后处理 |
|
||||
| HTML | 是 | 写入组装后的 HTML |
|
||||
| Markdown | 是 | 去除样式与脚本,再由 Turndown 转换 |
|
||||
| TXT | 是 | 转换 Markdown,再去除 Markdown 标记 |
|
||||
| EPUB | 否 | 独立从章节 HTML 打包,未接入该钩子 |
|
||||
|
||||
## 注入参数
|
||||
|
||||
脚本作为 `new Function("html", "document", "pages", "cfg", source)`
|
||||
的函数体执行。 这些变量由运行器注入,不需要也不能使用模块 `import` / `export`
|
||||
声明来获取:
|
||||
|
||||
| 名称 | 实际值 | 修改效果 |
|
||||
| ---------- | ---------------------------------------------------------------- | -------------------------------------------------------- |
|
||||
| `html` | 全部片段以换行拼接的字符串 | 仅重新赋值不会改变已经解析的 DOM;需要显式返回新的字符串 |
|
||||
| `document` | `DOMParser.parseFromString(html, "text/html")` 结果,可能为 null | 修改 DOM 会反映到默认序列化结果 |
|
||||
| `pages` | 组装片段数组的浅复制 | 直接改数组不会自动更新 DOM;需要自行合并并返回 |
|
||||
| `cfg` | 当前 DocuBuilder 的配置对象 | 可修改本次组装所使用的配置,传播边界见下文 |
|
||||
|
||||
优先使用运行时的 DOMParser;不可用时动态加载 deno_dom。它是 HTML 解析器,
|
||||
不是浏览器页面:这里没有排版坐标、最终页码、浏览器事件或可依赖的 `window`。
|
||||
最终页码应由 PDF 链接标记定位。
|
||||
|
||||
## DOM 修改与返回值
|
||||
|
||||
通常直接修改 `document`,无需 return:
|
||||
|
||||
```js
|
||||
if (document) {
|
||||
for (const image of document.querySelectorAll("img")) {
|
||||
image.setAttribute("loading", "eager");
|
||||
}
|
||||
const style = document.createElement("style");
|
||||
style.textContent = ".passages { overflow-wrap: anywhere; }";
|
||||
document.head?.appendChild(style);
|
||||
}
|
||||
```
|
||||
|
||||
脚本末尾由运行器追加:
|
||||
|
||||
```js
|
||||
return document?.documentElement?.outerHTML ?? html;
|
||||
```
|
||||
|
||||
也可显式返回完整 HTML 字符串,覆盖默认序列化,例如:
|
||||
|
||||
```js
|
||||
return html.replaceAll("旧刊名", "新刊名");
|
||||
```
|
||||
|
||||
自行返回时要保留布局、章节容器和 PDF 定位链接;返回非字符串没有运行时校验,
|
||||
会使后续输出行为不可靠。修改 `pages` 后需要 `return pages.join("\n")` 才能生效,
|
||||
这会绕过 `document` 上的修改。默认的 `outerHTML` 不包含原始 doctype。
|
||||
|
||||
## 配置修改与索引
|
||||
|
||||
现有 printable 模板设置:
|
||||
|
||||
```js
|
||||
cfg.indexSearch = { mode: "sequential" };
|
||||
```
|
||||
|
||||
索引默认优先读取临时文章链接,只有缺失标记时才使用全文搜索配置。 搜索支持
|
||||
`single_jump`、`sequential`、`window`;窗口模式可指定
|
||||
`beforePages`、`afterPages`。
|
||||
|
||||
新输出适配器会为 DocuBuilder 浅复制配置。顶层赋值(如 `cfg.indexSearch = ...`)
|
||||
不会自动同步回流水线原始配置;嵌套对象仍可能共享引用。因此不要依赖脚本修改顶层
|
||||
配置来控制 PDF 后续索引或后处理。需要可靠配置时,在调用 `PdfOutput` 前设置
|
||||
实际传入的配置;现有 printable 的脚本赋值在新适配器路径下存在这一传播限制。
|
||||
脚本适合处理 HTML,转换流程配置应由调用方明确传入。
|
||||
|
||||
PDF 中 `https://wpep.invalid/article-start-编号` 是定位用链接:不要删除、重排为
|
||||
与章节数组不一致的顺序,或将文章标题整体替换到丢失这些链接。目录合并阶段会清除
|
||||
临时文章链接,保留正常读者链接。未添加目录的 PDF 当前不会经过该清理阶段。
|
||||
|
||||
## 运行规则与验证
|
||||
|
||||
脚本同步执行,不支持顶层 `await`;不要返回 Promise。该钩子不是沙箱:它在 Deno
|
||||
进程内执行,具有进程已获准的能力。异常会中断导出,不会静默跳过。
|
||||
|
||||
先用小规模出版物验证 DOM 和最终输出,再检查 PDF 分页。对 HTML 的结构变更应确认:
|
||||
专栏标题与首篇文章关系、后续文章分页、图片显示、文章定位链接,以及普通链接仍正确。
|
||||
修改共享行为后在 `tools/gp` 运行:
|
||||
|
||||
```sh
|
||||
nix develop -c deno test -A
|
||||
nix develop -c deno check main.ts document/mod.ts pdf_builder/worker.ts
|
||||
```
|
||||
@@ -0,0 +1 @@
|
||||
<mxGraphModel adaptiveColors="auto" pageWidth="650" pageHeight="660"><root><mxCell id="0" /><mxCell id="1" parent="0" /><mxCell id="ir" value="HtmlDocument<br>章节 HTML 与布局" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#d9eee8;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="40" y="30" width="230" height="80" as="geometry" /></mxCell><mxCell id="assemble" value="DocuBuilder.as_html_page()<br>嵌入图片 · 组装 HTML · PDF 标记" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#d9eee8;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="40" y="160" width="230" height="80" as="geometry" /></mxCell><mxCell id="disabled" value="build_js_script = false<br>直接返回组装 HTML" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#fff0cf;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="360" y="160" width="230" height="80" as="geometry" /></mxCell><mxCell id="dom" value="启用脚本:读取文件并解析 DOM<br>注入 html / document / pages / cfg" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e5e4f5;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="40" y="290" width="230" height="80" as="geometry" /></mxCell><mxCell id="run" value="同步执行 build.js<br>修改 DOM 或显式 return 字符串" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e5e4f5;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="40" y="420" width="230" height="80" as="geometry" /></mxCell><mxCell id="serialize" value="默认序列化 documentElement<br>无 DOM 时回退 html" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#d9eee8;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="360" y="420" width="230" height="80" as="geometry" /></mxCell><mxCell id="output" value="继续导出 PDF / HTML / MD / TXT<br>EPUB 不经过此钩子" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#fff0cf;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="360" y="550" width="230" height="80" as="geometry" /></mxCell><mxCell id="e0" source="ir" target="assemble" edge="1" parent="1" style="edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;endArrow=block;"><mxGeometry relative="1" as="geometry" /></mxCell><mxCell id="e1" source="assemble" target="disabled" edge="1" parent="1" style="edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;endArrow=block;"><mxGeometry relative="1" as="geometry" /></mxCell><mxCell id="e2" source="assemble" target="dom" edge="1" parent="1" style="edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;endArrow=block;"><mxGeometry relative="1" as="geometry" /></mxCell><mxCell id="e3" source="dom" target="run" edge="1" parent="1" style="edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;endArrow=block;"><mxGeometry relative="1" as="geometry" /></mxCell><mxCell id="e4" source="run" target="serialize" edge="1" parent="1" style="edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;endArrow=block;"><mxGeometry relative="1" as="geometry" /></mxCell><mxCell id="e5" source="serialize" target="output" edge="1" parent="1" style="edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;endArrow=block;"><mxGeometry relative="1" as="geometry" /></mxCell><mxCell id="e6" source="disabled" target="output" edge="1" parent="1" style="edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;endArrow=block;"><mxGeometry relative="1" as="geometry" /></mxCell></root></mxGraphModel>
|
||||
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="650" height="660" viewBox="0 0 650 660" role="img"><title>build-js-workflow</title><metadata><mxGraphModel adaptiveColors="auto" pageWidth="650" pageHeight="660"><root><mxCell id="0" /><mxCell id="1" parent="0" /><mxCell id="ir" value="HtmlDocument&lt;br&gt;章节 HTML 与布局" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#d9eee8;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="40" y="30" width="230" height="80" as="geometry" /></mxCell><mxCell id="assemble" value="DocuBuilder.as_html_page()&lt;br&gt;嵌入图片 · 组装 HTML · PDF 标记" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#d9eee8;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="40" y="160" width="230" height="80" as="geometry" /></mxCell><mxCell id="disabled" value="build_js_script = false&lt;br&gt;直接返回组装 HTML" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#fff0cf;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="360" y="160" width="230" height="80" as="geometry" /></mxCell><mxCell id="dom" value="启用脚本:读取文件并解析 DOM&lt;br&gt;注入 html / document / pages / cfg" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e5e4f5;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="40" y="290" width="230" height="80" as="geometry" /></mxCell><mxCell id="run" value="同步执行 build.js&lt;br&gt;修改 DOM 或显式 return 字符串" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e5e4f5;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="40" y="420" width="230" height="80" as="geometry" /></mxCell><mxCell id="serialize" value="默认序列化 documentElement&lt;br&gt;无 DOM 时回退 html" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#d9eee8;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="360" y="420" width="230" height="80" as="geometry" /></mxCell><mxCell id="output" value="继续导出 PDF / HTML / MD / TXT&lt;br&gt;EPUB 不经过此钩子" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#fff0cf;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="360" y="550" width="230" height="80" as="geometry" /></mxCell><mxCell id="e0" source="ir" target="assemble" edge="1" parent="1" style="edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;endArrow=block;"><mxGeometry relative="1" as="geometry" /></mxCell><mxCell id="e1" source="assemble" target="disabled" edge="1" parent="1" style="edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;endArrow=block;"><mxGeometry relative="1" as="geometry" /></mxCell><mxCell id="e2" source="assemble" target="dom" edge="1" parent="1" style="edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;endArrow=block;"><mxGeometry relative="1" as="geometry" /></mxCell><mxCell id="e3" source="dom" target="run" edge="1" parent="1" style="edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;endArrow=block;"><mxGeometry relative="1" as="geometry" /></mxCell><mxCell id="e4" source="run" target="serialize" edge="1" parent="1" style="edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;endArrow=block;"><mxGeometry relative="1" as="geometry" /></mxCell><mxCell id="e5" source="serialize" target="output" edge="1" parent="1" style="edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;endArrow=block;"><mxGeometry relative="1" as="geometry" /></mxCell><mxCell id="e6" source="disabled" targLine truncated
|
||||
|
After Width: | Height: | Size: 8.9 KiB |
@@ -0,0 +1 @@
|
||||
<mxGraphModel adaptiveColors="auto" pageWidth="900" pageHeight="570"><root><mxCell id="0" /><mxCell id="1" parent="0" /><mxCell id="publication" value="PublicationInput<br>出版物 → 模板与 Markdown 渲染" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#d9eee8;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="40" y="30" width="230" height="80" as="geometry" /></mxCell><mxCell id="html" value="HtmlInput<br>已有 HTML 与章节元数据" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#d9eee8;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="330" y="30" width="230" height="80" as="geometry" /></mxCell><mxCell id="ir" value="HtmlDocument<br>章节 HTML · 元数据 · 资源 · 布局" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#bfe3d8;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="185" y="160" width="230" height="80" as="geometry" /></mxCell><mxCell id="text" value="TextOutput<br>组装 HTML · 图片嵌入 · build.js" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e5e4f5;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="40" y="300" width="230" height="80" as="geometry" /></mxCell><mxCell id="pdf" value="PdfOutput<br>组装 HTML · 标记 · build.js" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e5e4f5;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="330" y="300" width="230" height="80" as="geometry" /></mxCell><mxCell id="epub" value="EpubOutput<br>章节链接与资源打包" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e5e4f5;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="620" y="300" width="230" height="80" as="geometry" /></mxCell><mxCell id="textfile" value="HTML / Markdown / TXT<br>转换并写入文件" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#fff0cf;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="40" y="450" width="230" height="80" as="geometry" /></mxCell><mxCell id="pdffile" value="PDF<br>排版 → 索引 → 清理标记 → 后处理" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#fff0cf;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="330" y="450" width="230" height="80" as="geometry" /></mxCell><mxCell id="epubfile" value="EPUB<br>导航 · OPF · XHTML · ZIP" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#fff0cf;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="620" y="450" width="230" height="80" as="geometry" /></mxCell><mxCell id="e0" source="publication" target="ir" edge="1" parent="1" style="edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;endArrow=block;"><mxGeometry relative="1" as="geometry" /></mxCell><mxCell id="e1" source="html" target="ir" edge="1" parent="1" style="edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;endArrow=block;"><mxGeometry relative="1" as="geometry" /></mxCell><mxCell id="e2" source="ir" target="text" edge="1" parent="1" style="edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;endArrow=block;"><mxGeometry relative="1" as="geometry" /></mxCell><mxCell id="e3" source="ir" target="pdf" edge="1" parent="1" style="edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;endArrow=block;"><mxGeometry relative="1" as="geometry" /></mxCell><mxCell id="e4" source="ir" target="epub" edge="1" parent="1" style="edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;endArrow=block;"><mxGeometry relative="1" as="geometry" /></mxCell><mxCell id="e5" source="text" target="textfile" edge="1" parent="1" style="edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;endArrow=block;"><mxGeometry relative="1" as="geometry" /></mxCell><mxCell id="e6" source="pdf" target="pdffile" edge="1" parent="1" style="edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;endArrow=block;"><mxGeometry relative="1" as="geometry" /></mxCell><mxCell id="e7" source="epub" target="epubfile" edge="1" parent="1" style="edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;endArrow=block;"><mxGeometry relative="1" as="geometry" /></mxCell></root></mxGraphModel>
|
||||
@@ -0,0 +1 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" width="900" height="570" viewBox="0 0 900 570" role="img"><title>document-workflow</title><metadata><mxGraphModel adaptiveColors="auto" pageWidth="900" pageHeight="570"><root><mxCell id="0" /><mxCell id="1" parent="0" /><mxCell id="publication" value="PublicationInput&lt;br&gt;出版物 → 模板与 Markdown 渲染" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#d9eee8;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="40" y="30" width="230" height="80" as="geometry" /></mxCell><mxCell id="html" value="HtmlInput&lt;br&gt;已有 HTML 与章节元数据" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#d9eee8;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="330" y="30" width="230" height="80" as="geometry" /></mxCell><mxCell id="ir" value="HtmlDocument&lt;br&gt;章节 HTML · 元数据 · 资源 · 布局" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#bfe3d8;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="185" y="160" width="230" height="80" as="geometry" /></mxCell><mxCell id="text" value="TextOutput&lt;br&gt;组装 HTML · 图片嵌入 · build.js" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e5e4f5;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="40" y="300" width="230" height="80" as="geometry" /></mxCell><mxCell id="pdf" value="PdfOutput&lt;br&gt;组装 HTML · 标记 · build.js" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e5e4f5;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="330" y="300" width="230" height="80" as="geometry" /></mxCell><mxCell id="epub" value="EpubOutput&lt;br&gt;章节链接与资源打包" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#e5e4f5;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="620" y="300" width="230" height="80" as="geometry" /></mxCell><mxCell id="textfile" value="HTML / Markdown / TXT&lt;br&gt;转换并写入文件" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#fff0cf;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="40" y="450" width="230" height="80" as="geometry" /></mxCell><mxCell id="pdffile" value="PDF&lt;br&gt;排版 → 索引 → 清理标记 → 后处理" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#fff0cf;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="330" y="450" width="230" height="80" as="geometry" /></mxCell><mxCell id="epubfile" value="EPUB&lt;br&gt;导航 · OPF · XHTML · ZIP" style="rounded=1;whiteSpace=wrap;html=1;fillColor=#fff0cf;strokeColor=#52716b;fontSize=14;" vertex="1" parent="1"><mxGeometry x="620" y="450" width="230" height="80" as="geometry" /></mxCell><mxCell id="e0" source="publication" target="ir" edge="1" parent="1" style="edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;endArrow=block;"><mxGeometry relative="1" as="geometry" /></mxCell><mxCell id="e1" source="html" target="ir" edge="1" parent="1" style="edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;endArrow=block;"><mxGeometry relative="1" as="geometry" /></mxCell><mxCell id="e2" source="ir" target="text" edge="1" parent="1" style="edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;endArrow=block;"><mxGeometry relative="1" as="geometry" /></mxCell><mxCell id="e3" source="ir" target="pdf" edge="1" parent="1" style="edgeStyle=orthogonalEdgeStyle;rounded=1;html=1;endArrow=block;"><mxGLine truncated
|
||||
|
After Width: | Height: | Size: 11 KiB |
@@ -0,0 +1,179 @@
|
||||
# 文档输入输出接口开发指南
|
||||
|
||||
所有格式以 `HtmlDocument` 为交换数据。输入端负责读取源内容、渲染模板和生成章节
|
||||
HTML;输出端负责格式转换、资源处理与写入文件。公共入口是
|
||||
[`document/mod.ts`](../document/mod.ts)。
|
||||
|
||||

|
||||
|
||||
[编辑 draw.io 源图](diagrams/document-workflow.drawio)。SVG
|
||||
是文档预览,修改图后应同步更新预览。
|
||||
|
||||
## 接口契约
|
||||
|
||||
```ts
|
||||
interface DocumentInput {
|
||||
read(): Promise<HtmlDocument>;
|
||||
}
|
||||
|
||||
interface DocumentOutput {
|
||||
write(document: HtmlDocument): Promise<OutputResult>;
|
||||
}
|
||||
|
||||
interface OutputResult {
|
||||
readonly path: string;
|
||||
readonly mediaType: string;
|
||||
}
|
||||
```
|
||||
|
||||
`convertDocument(input, output)` 依次调用 `read()` 和 `write()`,返回写入结果。
|
||||
错误直接向调用方传播;输出端应在 `finally` 中释放浏览器等资源。
|
||||
接口没有流式读取、进度、取消或多文件返回协议;需要这些能力时应显式扩展契约。
|
||||
|
||||
## HTML 中间文档
|
||||
|
||||
完整类型见 [`document/types.ts`](../document/types.ts)。
|
||||
|
||||
| 字段 | 含义与约定 |
|
||||
| --------------- | ------------------------------------------------------------- |
|
||||
| `kind` | 固定为 `"html"` |
|
||||
| `metadata` | `title`、`author`、`language`,可选 `identifier` |
|
||||
| `baseDirectory` | 出版物绝对目录,供资源边界和路径解析使用 |
|
||||
| `assetBase` | 共享资源目录;`PublicationInput` 会将其规范为绝对文件系统路径 |
|
||||
| `chapters` | 已排序的章节数组;数组顺序就是输出顺序 |
|
||||
| `layout` | `cover`、`styles`、`before`、`after` 四个 HTML 字符串 |
|
||||
|
||||
每个 `HtmlChapter` 包含唯一 `id`、`title`、`html`、绝对 `sourcePath`、文章
|
||||
`index`,以及 `column: { name, index }`。`html` 是章节片段,保留原始资源引用,
|
||||
不含 PDF 临时定位链接。`sourcePath` 为 EPUB 相对图片、章节链接提供解析基址,
|
||||
即使输入并非 Markdown,也应提供有意义的来源路径。
|
||||
|
||||
`readonly` 是 TypeScript
|
||||
的编译期约束,不会在运行时冻结对象。输出实现不得修改输入,
|
||||
需要加工时建立局部字符串或复制数据。同一份文档可顺序交给多个输出端。 当前 PDF
|
||||
标记编号仍按章节数组的顺序生成,尚未直接使用 `chapter.id`;标题索引仍以 标题为
|
||||
Map 键,因此不同文章重名仍是现有索引的限制。
|
||||
|
||||
## 使用现有输入输出
|
||||
|
||||
以下示例放在 `tools/gp/` 下运行,`config` 和 `variables` 由调用方提供:
|
||||
|
||||
```ts
|
||||
import {
|
||||
convertDocument,
|
||||
EpubOutput,
|
||||
PublicationInput,
|
||||
TextOutput,
|
||||
} from "./document/mod.ts";
|
||||
|
||||
await convertDocument(
|
||||
new PublicationInput(config, variables),
|
||||
new TextOutput(config, "html"),
|
||||
);
|
||||
|
||||
// 复用同一份 HTML,避免重复读取和渲染。
|
||||
const document = await new PublicationInput(config, variables).read();
|
||||
await new TextOutput(config, "md").write(document);
|
||||
await new EpubOutput(config).write(document);
|
||||
```
|
||||
|
||||
| 实现 | 处理方式 |
|
||||
| -------------------------------------------------- | -------------------------------------------------------------------- |
|
||||
| `PublicationInput(config, variables)` | 使用 PFiles 排序并读取出版物;渲染模板和 Markdown,生成章节 HTML |
|
||||
| `HtmlInput(document)` | 直接返回调用方传入的中间文档,不读取出版物 |
|
||||
| `TextOutput(config, "html" / "md" / "txt", path?)` | 组装 HTML、嵌入本地章节图片、执行 build.js,按需转为 Markdown 或 TXT |
|
||||
| `PdfOutput(config, options?)` | 浏览器排版、可选目录、清理定位标记、书签、可选 qpdf 和签名 |
|
||||
| `EpubOutput(config, path?)` | 从章节 HTML 生成 XHTML,重写章节链接、打包本地资源并生成 EPUB 导航 |
|
||||
|
||||
PDF 配置类型为 `GPConfig`,扩展共享 `DocumentConfig`,包含 `pdf` 和
|
||||
`playwright`;导出选项见 `PDFPipelineOptions`。CLI 的 PDF Worker
|
||||
使用同一组接口。 旧 `DocuBuilder` / `EpubBuilder`
|
||||
方法保留作兼容入口,新扩展优先使用公共接口。
|
||||
|
||||
输出路径可使用 `md.path`、`html.path`、`txt.path`、`epub.path` 或
|
||||
`playwright.path`。HTML/TXT 未单独配置时从 `md.path` 或 `markdown_output`
|
||||
派生,EPUB 也有兼容回退路径。转换器本身不选择路径,由输出适配器负责。
|
||||
|
||||
## 实现新的输入
|
||||
|
||||
例如接入数据库、远程接口或已有 HTML 文件时,实现 `read()` 并生成章节结构:
|
||||
|
||||
```ts
|
||||
import type { DocumentInput, HtmlDocument } from "./document/mod.ts";
|
||||
|
||||
class ArticleHtmlInput implements DocumentInput {
|
||||
constructor(private readonly root: string) {}
|
||||
|
||||
async read(): Promise<HtmlDocument> {
|
||||
return {
|
||||
kind: "html",
|
||||
metadata: { title: "示例刊物", author: "编辑部", language: "zh-CN" },
|
||||
baseDirectory: this.root,
|
||||
assetBase: `${this.root}/assets`,
|
||||
chapters: [{
|
||||
id: "article-1",
|
||||
title: "示例文章",
|
||||
html: await Deno.readTextFile(`${this.root}/article.html`),
|
||||
sourcePath: `${this.root}/article.html`,
|
||||
index: 1,
|
||||
column: { name: "示例专栏", index: 1 },
|
||||
}],
|
||||
layout: { cover: "", styles: "", before: "", after: "" },
|
||||
};
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
`root` 应为绝对路径。输入负责确认章节顺序、ID
|
||||
唯一性和元数据完整性;当前公共流水线 不提供运行时 schema
|
||||
校验。模板变量应在输入阶段解析一次,不要让新输出再次解释源模板。
|
||||
|
||||
## 实现新的输出
|
||||
|
||||
新输出只依赖中间文档,不应重新读取出版物 Markdown:
|
||||
|
||||
```ts
|
||||
import type {
|
||||
DocumentOutput,
|
||||
HtmlDocument,
|
||||
OutputResult,
|
||||
} from "./document/mod.ts";
|
||||
|
||||
class ChapterJsonOutput implements DocumentOutput {
|
||||
constructor(private readonly path: string) {}
|
||||
|
||||
async write(document: HtmlDocument): Promise<OutputResult> {
|
||||
await Deno.writeTextFile(
|
||||
this.path,
|
||||
JSON.stringify(document.chapters, null, 2),
|
||||
);
|
||||
return { path: this.path, mediaType: "application/json" };
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
此示例要求父目录已存在。生产实现应负责创建输出目录、处理资源路径、释放资源,
|
||||
并将失败抛给调用方。如果需要 CLI 子命令,再接入 `cli/handlers.ts` 和
|
||||
`cli/match.ts`; 无需调整其他输出实现。
|
||||
|
||||
## 资源与脚本边界
|
||||
|
||||
本地章节图片在文本和 PDF 组装阶段按共享资源基址嵌入;远程 URL 保持原样。 EPUB
|
||||
根据章节来源路径和共享资源目录打包本地文件,远程资源当前不自动下载。
|
||||
封面、字体和附加布局 HTML 不一定经过相同的章节图片嵌入逻辑。
|
||||
|
||||
EPUB 当前只消费章节和元数据,不复用 PDF 布局,也不执行 build.js。
|
||||
PDF、HTML、Markdown、TXT 的脚本阶段详见 [build.js 开发指南](build-js.md)。
|
||||
|
||||
## 验证
|
||||
|
||||
在 `tools/gp/` 运行:
|
||||
|
||||
```sh
|
||||
nix develop -c deno test -A document/document_test.ts
|
||||
nix develop -c deno check document/mod.ts main.ts pdf_builder/worker.ts
|
||||
```
|
||||
|
||||
新增输入应验证排序、变量与资源基址;新增输出应使用内存构造的 `HtmlDocument`
|
||||
验证内容、路径、媒体类型以及输入未被修改。PDF / EPUB
|
||||
的资源和索引变更还应运行相应测试。
|
||||
@@ -0,0 +1,102 @@
|
||||
import { assertEquals, assertStringIncludes } from "@std/assert";
|
||||
import {
|
||||
convertDocument,
|
||||
EpubOutput,
|
||||
type HtmlDocument,
|
||||
HtmlInput,
|
||||
PdfOutput,
|
||||
PublicationInput,
|
||||
TextOutput,
|
||||
textOutputPath,
|
||||
} from "./mod.ts";
|
||||
import { inspectEpub } from "../epub_builder/epub_builder.ts";
|
||||
|
||||
Deno.test("publication input renders chapter HTML once and preserves metadata", async () => {
|
||||
const root = await Deno.makeTempDir();
|
||||
try {
|
||||
await Deno.mkdir(`${root}/1::Column`);
|
||||
await Deno.writeTextFile(
|
||||
`${root}/1::Column/1::Title.md`,
|
||||
"# {{ document.title }}\n\n{{ greeting }}",
|
||||
);
|
||||
const document = await new PublicationInput({
|
||||
journal: {
|
||||
from_directory: root,
|
||||
template_directory: root,
|
||||
build_mode: "detached",
|
||||
build_js_script: false,
|
||||
},
|
||||
}, { greeting: "Hello" }).read();
|
||||
assertEquals(document.chapters[0].id, "article-1");
|
||||
assertEquals(document.chapters[0].column.name, "Column");
|
||||
assertStringIncludes(document.chapters[0].html, "Hello");
|
||||
assertEquals(document.chapters[0].html.includes("article-start-"), false);
|
||||
} finally {
|
||||
await Deno.remove(root, { recursive: true });
|
||||
}
|
||||
});
|
||||
|
||||
Deno.test("HTML input exports text and EPUB without reading publication sources", async () => {
|
||||
const root = await Deno.makeTempDir();
|
||||
const document: HtmlDocument = {
|
||||
kind: "html",
|
||||
metadata: { title: "Book", author: "Author", language: "en" },
|
||||
baseDirectory: root,
|
||||
assetBase: root,
|
||||
chapters: [{
|
||||
id: "article-1",
|
||||
title: "Title",
|
||||
index: 1,
|
||||
sourcePath: `${root}/missing.md`,
|
||||
column: { name: "Column", index: 1 },
|
||||
html: "<h1>Title</h1><p>Hello <strong>world</strong></p>",
|
||||
}],
|
||||
layout: { cover: "", styles: "", before: "", after: "" },
|
||||
};
|
||||
const original = JSON.stringify(document);
|
||||
const config = {
|
||||
journal: {
|
||||
from_directory: `${root}/does-not-exist`,
|
||||
build_js_script: false as const,
|
||||
},
|
||||
md: { path: `${root}/book.md` },
|
||||
};
|
||||
try {
|
||||
for (const format of ["html", "md", "txt"] as const) {
|
||||
const result = await convertDocument(
|
||||
new HtmlInput(document),
|
||||
new TextOutput(config, format),
|
||||
);
|
||||
const content = await Deno.readTextFile(result.path);
|
||||
assertStringIncludes(content, "Hello");
|
||||
assertEquals(content.includes("article-start-"), false);
|
||||
}
|
||||
const result = await convertDocument(
|
||||
new HtmlInput(document),
|
||||
new EpubOutput(config),
|
||||
);
|
||||
const archive = inspectEpub(await Deno.readFile(result.path));
|
||||
assertStringIncludes(
|
||||
new TextDecoder().decode(archive["EPUB/text/0001-title.xhtml"]),
|
||||
"Hello",
|
||||
);
|
||||
assertEquals(JSON.stringify(document), original);
|
||||
assertEquals(
|
||||
textOutputPath({ ...config, html: { path: "custom.html" } }, "html"),
|
||||
"custom.html",
|
||||
);
|
||||
if (Deno.env.get("CHROMIUM_PATH")) {
|
||||
const pdf = await convertDocument(
|
||||
new HtmlInput(document),
|
||||
new PdfOutput({
|
||||
...config,
|
||||
pdf: { generate_index: false },
|
||||
playwright: { path: `${root}/book.pdf`, format: "A4" },
|
||||
}),
|
||||
);
|
||||
assertEquals((await Deno.stat(pdf.path)).size > 0, true);
|
||||
}
|
||||
} finally {
|
||||
await Deno.remove(root, { recursive: true });
|
||||
}
|
||||
});
|
||||
@@ -0,0 +1,77 @@
|
||||
import { resolve } from "node:path";
|
||||
import { fileURLToPath } from "node:url";
|
||||
import { PFiles } from "../publication/file.ts";
|
||||
import { md } from "../mdit/md.ts";
|
||||
import { renderTemplate, type TemplateVariables } from "../template/engine.ts";
|
||||
import type { DocumentConfig, DocumentInput, HtmlDocument } from "./types.ts";
|
||||
|
||||
export function publicationToHtml(
|
||||
files: PFiles,
|
||||
config: DocumentConfig,
|
||||
variables: TemplateVariables = {},
|
||||
): HtmlDocument {
|
||||
const render = (source: string) => renderTemplate(source, variables);
|
||||
let id = 0;
|
||||
return {
|
||||
kind: "html",
|
||||
metadata: {
|
||||
title: config.journal.title ?? "未命名出版物",
|
||||
author: config.journal.author ?? "未知作者",
|
||||
language: config.journal.language ?? "zh-CN",
|
||||
identifier: config.journal.identifier,
|
||||
},
|
||||
baseDirectory: resolve(config.journal.from_directory),
|
||||
assetBase: /^file:\/\//i.test(config.asset_base ?? "")
|
||||
? fileURLToPath(config.asset_base!)
|
||||
: resolve(config.asset_base ?? "../../assets"),
|
||||
chapters: files.pcolumes.flatMap((column) =>
|
||||
column.pfiles.map((file) => ({
|
||||
id: `article-${++id}`,
|
||||
title: file.meta.title,
|
||||
sourcePath: resolve(file.full_name),
|
||||
index: file.meta.index,
|
||||
column: { name: column.name, index: column.index },
|
||||
html: md.render(renderTemplate(file.content, {
|
||||
...variables,
|
||||
document: {
|
||||
title: file.meta.title,
|
||||
index: file.meta.index,
|
||||
file: file.full_name,
|
||||
},
|
||||
column: { name: column.name, index: column.index },
|
||||
})),
|
||||
}))
|
||||
),
|
||||
layout: {
|
||||
cover: render(files.cover_html ?? ""),
|
||||
styles: render(files.style_html ?? ""),
|
||||
before: render(files.other_htmls.join("\n")),
|
||||
after: render(files.end_html ?? ""),
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
export class PublicationInput implements DocumentInput {
|
||||
constructor(
|
||||
private readonly config: DocumentConfig,
|
||||
private readonly variables: TemplateVariables = {},
|
||||
) {}
|
||||
|
||||
async read(): Promise<HtmlDocument> {
|
||||
const files = new PFiles(
|
||||
this.config.journal.build_mode ?? "flat",
|
||||
this.config.journal.from_directory,
|
||||
this.config,
|
||||
);
|
||||
return publicationToHtml(files, this.config, this.variables);
|
||||
}
|
||||
}
|
||||
|
||||
/** Allows callers and future parsers to supply HTML without publication files. */
|
||||
export class HtmlInput implements DocumentInput {
|
||||
constructor(private readonly document: HtmlDocument) {}
|
||||
|
||||
async read(): Promise<HtmlDocument> {
|
||||
return this.document;
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,8 @@
|
||||
export * from "./types.ts";
|
||||
export { HtmlInput, PublicationInput } from "./inputs.ts";
|
||||
export {
|
||||
EpubOutput,
|
||||
PdfOutput,
|
||||
TextOutput,
|
||||
textOutputPath,
|
||||
} from "./outputs.ts";
|
||||
@@ -0,0 +1,96 @@
|
||||
import { dirname } from "node:path";
|
||||
import { DocuBuilder } from "../pdf_builder/docu_builder.ts";
|
||||
import { EpubBuilder } from "../epub_builder/epub_builder.ts";
|
||||
import {
|
||||
type GPConfig,
|
||||
type PDFPipelineOptions,
|
||||
processPdfConfig,
|
||||
} from "../pdf_builder/pipeline.ts";
|
||||
import type {
|
||||
DocumentConfig,
|
||||
DocumentOutput,
|
||||
HtmlDocument,
|
||||
OutputResult,
|
||||
} from "./types.ts";
|
||||
|
||||
export type TextFormat = "html" | "md" | "txt";
|
||||
|
||||
export function textOutputPath(
|
||||
config: DocumentConfig,
|
||||
format: TextFormat,
|
||||
): string {
|
||||
const explicit = format === "html"
|
||||
? config.html?.path
|
||||
: format === "txt"
|
||||
? config.txt?.path
|
||||
: config.md?.path;
|
||||
if (explicit) return explicit;
|
||||
const source = config.md?.path ?? config.markdown_output;
|
||||
if (!source) {
|
||||
throw new Error(
|
||||
`Missing ${format} output path. Use ${format}.path or md.path.`,
|
||||
);
|
||||
}
|
||||
return format === "md"
|
||||
? source
|
||||
: /\.md$/i.test(source)
|
||||
? source.replace(/\.md$/i, `.${format}`)
|
||||
: `${source}.${format}`;
|
||||
}
|
||||
|
||||
export class TextOutput implements DocumentOutput {
|
||||
constructor(
|
||||
private readonly config: DocumentConfig,
|
||||
private readonly format: TextFormat,
|
||||
private readonly path = textOutputPath(config, format),
|
||||
) {}
|
||||
|
||||
async write(document: HtmlDocument): Promise<OutputResult> {
|
||||
const builder = new DocuBuilder({
|
||||
...this.config,
|
||||
asset_base: document.assetBase,
|
||||
});
|
||||
let content: string;
|
||||
if (this.format === "html") {
|
||||
content = await builder.as_html_page(document, { articleMarkers: false });
|
||||
} else {
|
||||
if (this.format === "md") await builder.as_markdown(document);
|
||||
else await builder.as_txt(true, document);
|
||||
content = builder.pages.join("\n");
|
||||
}
|
||||
await Deno.mkdir(dirname(this.path), { recursive: true });
|
||||
await Deno.writeTextFile(this.path, content);
|
||||
return {
|
||||
path: this.path,
|
||||
mediaType: this.format === "html"
|
||||
? "text/html"
|
||||
: this.format === "md"
|
||||
? "text/markdown"
|
||||
: "text/plain",
|
||||
};
|
||||
}
|
||||
}
|
||||
|
||||
export class EpubOutput implements DocumentOutput {
|
||||
constructor(
|
||||
private readonly config: DocumentConfig,
|
||||
private readonly path?: string,
|
||||
) {}
|
||||
async write(document: HtmlDocument): Promise<OutputResult> {
|
||||
const builder = new EpubBuilder(this.config);
|
||||
const path = this.path ?? builder.outputPath();
|
||||
await builder.build(path, document);
|
||||
return { path, mediaType: "application/epub+zip" };
|
||||
}
|
||||
}
|
||||
|
||||
export class PdfOutput implements DocumentOutput {
|
||||
constructor(
|
||||
private readonly config: GPConfig,
|
||||
private readonly options: PDFPipelineOptions = {},
|
||||
) {}
|
||||
async write(document: HtmlDocument): Promise<OutputResult> {
|
||||
await processPdfConfig(this.config, this.options, document);
|
||||
return { path: this.config.playwright.path, mediaType: "application/pdf" };
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,73 @@
|
||||
import type { TemplateVariables } from "../template/engine.ts";
|
||||
|
||||
/** Configuration shared by publication inputs and output adapters. */
|
||||
export interface DocumentConfig {
|
||||
journal: {
|
||||
from_directory: string;
|
||||
template_directory?: string;
|
||||
build_mode?: "detached" | "flat";
|
||||
build_js_script?: string | false;
|
||||
title?: string;
|
||||
author?: string;
|
||||
language?: string;
|
||||
identifier?: string;
|
||||
};
|
||||
asset_base?: string;
|
||||
vars?: TemplateVariables;
|
||||
md?: { path?: string };
|
||||
html?: { path?: string };
|
||||
txt?: { path?: string };
|
||||
epub?: { path?: string; title?: string; author?: string; language?: string };
|
||||
markdown_output?: string;
|
||||
}
|
||||
|
||||
/** An HTML fragment with stable identity and its original resource base. */
|
||||
export interface HtmlChapter {
|
||||
readonly id: string;
|
||||
readonly title: string;
|
||||
readonly html: string;
|
||||
readonly sourcePath: string;
|
||||
readonly index: number;
|
||||
readonly column: { readonly name: string; readonly index: number };
|
||||
}
|
||||
|
||||
/** Portable intermediate representation. HTML fragments contain no PDF markers. */
|
||||
export interface HtmlDocument {
|
||||
readonly kind: "html";
|
||||
readonly metadata: {
|
||||
readonly title: string;
|
||||
readonly author: string;
|
||||
readonly language: string;
|
||||
readonly identifier?: string;
|
||||
};
|
||||
readonly baseDirectory: string;
|
||||
readonly assetBase: string;
|
||||
readonly chapters: readonly HtmlChapter[];
|
||||
readonly layout: {
|
||||
readonly cover: string;
|
||||
readonly styles: string;
|
||||
readonly before: string;
|
||||
readonly after: string;
|
||||
};
|
||||
}
|
||||
|
||||
export interface DocumentInput {
|
||||
read(): Promise<HtmlDocument>;
|
||||
}
|
||||
|
||||
export interface OutputResult {
|
||||
readonly path: string;
|
||||
readonly mediaType: string;
|
||||
}
|
||||
|
||||
export interface DocumentOutput {
|
||||
write(document: HtmlDocument): Promise<OutputResult>;
|
||||
}
|
||||
|
||||
/** The pipeline has no knowledge of publication files or output formats. */
|
||||
export async function convertDocument(
|
||||
input: DocumentInput,
|
||||
output: DocumentOutput,
|
||||
): Promise<OutputResult> {
|
||||
return await output.write(await input.read());
|
||||
}
|
||||
@@ -1,25 +1,11 @@
|
||||
import { strToU8, unzipSync, type Zippable, zipSync } from "fflate";
|
||||
import { dirname, extname, relative, resolve, sep } from "node:path";
|
||||
import { PFiles } from "./file.ts";
|
||||
import { md } from "./mdit/md.ts";
|
||||
import { renderTemplate, type TemplateVariables } from "./template_engine.ts";
|
||||
import { PFiles } from "../publication/file.ts";
|
||||
import type { TemplateVariables } from "../template/engine.ts";
|
||||
import { publicationToHtml } from "../document/inputs.ts";
|
||||
import type { DocumentConfig, HtmlDocument } from "../document/types.ts";
|
||||
|
||||
type EpubConfig = {
|
||||
journal: {
|
||||
build_mode?: "detached" | "flat";
|
||||
from_directory: string;
|
||||
template_directory: string;
|
||||
title?: string;
|
||||
author?: string;
|
||||
language?: string;
|
||||
identifier?: string;
|
||||
};
|
||||
vars?: Record<string, unknown>;
|
||||
epub?: { path?: string; title?: string; author?: string; language?: string };
|
||||
md?: { path?: string };
|
||||
markdown_output?: string;
|
||||
asset_base?: string;
|
||||
};
|
||||
type EpubConfig = DocumentConfig;
|
||||
|
||||
type EpubChapter = {
|
||||
id: string;
|
||||
@@ -188,115 +174,112 @@ export class EpubBuilder {
|
||||
return this;
|
||||
}
|
||||
|
||||
async build(path = this.outputPath()): Promise<void> {
|
||||
if (!this.pfiles) this.with_files_sync();
|
||||
async build(
|
||||
path = this.outputPath(),
|
||||
document?: HtmlDocument,
|
||||
): Promise<void> {
|
||||
if (!document && !this.pfiles) this.with_files_sync();
|
||||
const input = document ??
|
||||
publicationToHtml(this.pfiles!, this.config, this.variables);
|
||||
const chapters: EpubChapter[] = [];
|
||||
const mediaManifest = new Map<string, string>();
|
||||
const epubMedia = new Map<string, Uint8Array>();
|
||||
const allFiles = this.pfiles!;
|
||||
let index = 0;
|
||||
const sourceToHref = new Map<string, string>();
|
||||
for (const column of allFiles.pcolumes) {
|
||||
for (const pfile of column.pfiles) {
|
||||
index += 1;
|
||||
sourceToHref.set(
|
||||
resolve(pfile.full_name),
|
||||
`text/${slug(pfile.meta.title, index)}.xhtml`,
|
||||
);
|
||||
}
|
||||
for (const chapter of input.chapters) {
|
||||
index += 1;
|
||||
sourceToHref.set(
|
||||
resolve(chapter.sourcePath),
|
||||
`text/${slug(chapter.title, index)}.xhtml`,
|
||||
);
|
||||
}
|
||||
index = 0;
|
||||
for (const column of allFiles.pcolumes) {
|
||||
for (const pfile of column.pfiles) {
|
||||
index += 1;
|
||||
const id = `chapter-${index}`;
|
||||
const href = `text/${slug(pfile.meta.title, index)}.xhtml`;
|
||||
let html = md.render(renderTemplate(pfile.content, {
|
||||
...this.variables,
|
||||
document: {
|
||||
title: pfile.meta.title,
|
||||
index: pfile.meta.index,
|
||||
file: pfile.full_name,
|
||||
},
|
||||
column: { name: column.name, index: column.index },
|
||||
}));
|
||||
html = removeRepeatedLeadingTitle(html, pfile.meta.title);
|
||||
const targetDir = dirname(resolve(href));
|
||||
const sourceDir = dirname(resolve(pfile.full_name));
|
||||
const media: Array<{ href: string; bytes: Uint8Array; mime: string }> =
|
||||
[];
|
||||
const mediaBySource = new Map<string, string>();
|
||||
const knownTargets = new Set(sourceToHref.keys());
|
||||
const assetReferences = [
|
||||
...html.matchAll(/\b(src|href)=(['"])([^'"]+)\2/gi),
|
||||
];
|
||||
for (const reference of assetReferences) {
|
||||
const sourceUrl = reference[3];
|
||||
if (sourceUrl.split(/[?#]/, 1)[0].toLowerCase().endsWith(".md")) {
|
||||
continue;
|
||||
}
|
||||
if (/^(?:[a-z][a-z0-9+.-]*:|#|\/\/)/i.test(sourceUrl)) continue;
|
||||
const sourceAsset = resolve(
|
||||
sourceDir,
|
||||
decodeURIComponent(sourceUrl.split(/[?#]/, 1)[0]),
|
||||
);
|
||||
if (knownTargets.has(sourceAsset)) continue;
|
||||
if (
|
||||
!sourceAsset.startsWith(
|
||||
`${resolve(this.config.journal.from_directory)}${sep}`,
|
||||
)
|
||||
) continue;
|
||||
try {
|
||||
const info = await Deno.stat(sourceAsset);
|
||||
if (!info.isFile) continue;
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
const destination = mediaBySource.get(sourceAsset) ??
|
||||
`media/${index}/${media.length + 1}${
|
||||
extname(sourceAsset).toLowerCase()
|
||||
}`;
|
||||
if (!mediaBySource.has(sourceAsset)) {
|
||||
mediaBySource.set(sourceAsset, destination);
|
||||
const ext = extname(sourceAsset).toLowerCase();
|
||||
media.push({
|
||||
href: destination,
|
||||
bytes: await Deno.readFile(sourceAsset),
|
||||
mime: mediaType(ext),
|
||||
});
|
||||
}
|
||||
const escapedUrl = reference[0];
|
||||
const replacement = `${reference[1]}=${reference[2]}${
|
||||
relative(targetDir, resolve(destination)).split(sep).join("/")
|
||||
}${reference[2]}`;
|
||||
html = html.replace(escapedUrl, replacement);
|
||||
for (const chapter of input.chapters) {
|
||||
index += 1;
|
||||
const id = `chapter-${index}`;
|
||||
const href = `text/${slug(chapter.title, index)}.xhtml`;
|
||||
let html = removeRepeatedLeadingTitle(chapter.html, chapter.title);
|
||||
const targetDir = dirname(resolve(href));
|
||||
const sourceDir = dirname(resolve(chapter.sourcePath));
|
||||
const media: Array<{ href: string; bytes: Uint8Array; mime: string }> =
|
||||
[];
|
||||
const mediaBySource = new Map<string, string>();
|
||||
const knownTargets = new Set(sourceToHref.keys());
|
||||
const assetReferences = [
|
||||
...html.matchAll(/\b(src|href)=(['"])([^'"]+)\2/gi),
|
||||
];
|
||||
for (const reference of assetReferences) {
|
||||
const sourceUrl = reference[3];
|
||||
if (sourceUrl.split(/[?#]/, 1)[0].toLowerCase().endsWith(".md")) {
|
||||
continue;
|
||||
}
|
||||
// Resolve authored links that point to another Markdown chapter inside this EPUB.
|
||||
html = html.replace(
|
||||
/\b(href)=(['"])([^'"]+\.md(?:#[^'"]*)?)\2/gi,
|
||||
(match, attr: string, quote: string, target: string) => {
|
||||
const [filePart, fragment] = target.split("#", 2);
|
||||
const targetPath = resolve(sourceDir, decodeURIComponent(filePart));
|
||||
const chapterHref = sourceToHref.get(targetPath);
|
||||
return chapterHref
|
||||
? `${attr}=${quote}${
|
||||
relative(targetDir, resolve(chapterHref)).split(sep).join("/")
|
||||
}${fragment ? `#${fragment}` : ""}${quote}`
|
||||
: match;
|
||||
},
|
||||
if (/^(?:[a-z][a-z0-9+.-]*:|#|\/\/)/i.test(sourceUrl)) continue;
|
||||
const sourceAsset = resolve(
|
||||
sourceDir,
|
||||
decodeURIComponent(sourceUrl.split(/[?#]/, 1)[0]),
|
||||
);
|
||||
chapters.push({
|
||||
id,
|
||||
href,
|
||||
title: pfile.meta.title,
|
||||
html,
|
||||
sourcePath: pfile.full_name,
|
||||
column: column.name,
|
||||
});
|
||||
for (const asset of media) {
|
||||
epubMedia.set(asset.href, asset.bytes);
|
||||
mediaManifest.set(asset.href, asset.mime);
|
||||
const assetMarker = sourceUrl.match(/(?:^|\/)assets\/(.+)$/i);
|
||||
const sharedAsset = assetMarker && input.assetBase
|
||||
? resolve(input.assetBase, assetMarker[1])
|
||||
: undefined;
|
||||
const resolvedAsset = sharedAsset && await Deno.stat(sharedAsset)
|
||||
.then((info) => info.isFile ? sharedAsset : undefined)
|
||||
.catch(() => undefined) || sourceAsset;
|
||||
if (knownTargets.has(sourceAsset)) continue;
|
||||
const localAsset = resolvedAsset.startsWith(
|
||||
`${input.baseDirectory}${sep}`,
|
||||
) || (sharedAsset != null && resolvedAsset === sharedAsset);
|
||||
if (!localAsset) continue;
|
||||
try {
|
||||
const info = await Deno.stat(resolvedAsset);
|
||||
if (!info.isFile) continue;
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
const destination = mediaBySource.get(resolvedAsset) ??
|
||||
`media/${index}/${media.length + 1}${
|
||||
extname(resolvedAsset).toLowerCase()
|
||||
}`;
|
||||
if (!mediaBySource.has(resolvedAsset)) {
|
||||
mediaBySource.set(resolvedAsset, destination);
|
||||
const ext = extname(resolvedAsset).toLowerCase();
|
||||
media.push({
|
||||
href: destination,
|
||||
bytes: await Deno.readFile(resolvedAsset),
|
||||
mime: mediaType(ext),
|
||||
});
|
||||
}
|
||||
const escapedUrl = reference[0];
|
||||
const replacement = `${reference[1]}=${reference[2]}${
|
||||
relative(targetDir, resolve(destination)).split(sep).join("/")
|
||||
}${reference[2]}`;
|
||||
html = html.replace(escapedUrl, replacement);
|
||||
}
|
||||
// Resolve authored links that point to another Markdown chapter inside this EPUB.
|
||||
html = html.replace(
|
||||
/\b(href)=(['"])([^'"]+\.md(?:#[^'"]*)?)\2/gi,
|
||||
(match, attr: string, quote: string, target: string) => {
|
||||
const [filePart, fragment] = target.split("#", 2);
|
||||
const targetPath = resolve(sourceDir, decodeURIComponent(filePart));
|
||||
const chapterHref = sourceToHref.get(targetPath);
|
||||
return chapterHref
|
||||
? `${attr}=${quote}${
|
||||
relative(targetDir, resolve(chapterHref)).split(sep).join("/")
|
||||
}${fragment ? `#${fragment}` : ""}${quote}`
|
||||
: match;
|
||||
},
|
||||
);
|
||||
chapters.push({
|
||||
id,
|
||||
href,
|
||||
title: chapter.title,
|
||||
html,
|
||||
sourcePath: chapter.sourcePath,
|
||||
column: chapter.column.name,
|
||||
});
|
||||
for (const asset of media) {
|
||||
epubMedia.set(asset.href, asset.bytes);
|
||||
mediaManifest.set(asset.href, asset.mime);
|
||||
}
|
||||
}
|
||||
if (chapters.length === 0) {
|
||||
@@ -304,10 +287,10 @@ export class EpubBuilder {
|
||||
}
|
||||
|
||||
const epub = this.config.epub ?? {};
|
||||
const title = epub.title ?? this.config.journal.title ?? "未命名出版物";
|
||||
const author = epub.author ?? this.config.journal.author ?? "未知作者";
|
||||
const language = epub.language ?? this.config.journal.language ?? "zh-CN";
|
||||
const identifier = this.config.journal.identifier ??
|
||||
const title = epub.title ?? input.metadata.title;
|
||||
const author = epub.author ?? input.metadata.author;
|
||||
const language = epub.language ?? input.metadata.language;
|
||||
const identifier = input.metadata.identifier ??
|
||||
`urn:uuid:${await this.contentIdentifier(chapters)}`;
|
||||
const files: Record<string, Uint8Array> = {
|
||||
mimetype: strToU8("application/epub+zip"),
|
||||
@@ -79,3 +79,87 @@ Deno.test("uses EPUB output path or a stable default", () => {
|
||||
});
|
||||
assertEquals(builder.outputPath(), "./outputs/book.epub");
|
||||
});
|
||||
|
||||
Deno.test("bundles shared assets referenced by publication articles", async () => {
|
||||
const root = await Deno.makeTempDir();
|
||||
const publication = `${root}/publication`;
|
||||
const templates = `${root}/templates`;
|
||||
const assets = `${root}/assets`;
|
||||
const output = `${root}/shared-assets.epub`;
|
||||
await Deno.mkdir(`${publication}/1::专栏`, { recursive: true });
|
||||
await Deno.mkdir(templates, { recursive: true });
|
||||
await Deno.mkdir(`${assets}/pictures`, { recursive: true });
|
||||
await Deno.writeTextFile(
|
||||
`${publication}/1::专栏/1::文章.md`,
|
||||
"",
|
||||
);
|
||||
const imageBytes = new Uint8Array([137, 80, 78, 71, 13, 10, 26, 10]);
|
||||
await Deno.writeFile(`${assets}/pictures/shared.png`, imageBytes);
|
||||
const builder = new EpubBuilder({
|
||||
asset_base: assets,
|
||||
journal: {
|
||||
build_mode: "detached",
|
||||
from_directory: publication,
|
||||
template_directory: templates,
|
||||
},
|
||||
epub: { path: output },
|
||||
}).with_files_sync();
|
||||
|
||||
try {
|
||||
await builder.build();
|
||||
const archive = inspectEpub(await Deno.readFile(output));
|
||||
const chapter = new TextDecoder().decode(
|
||||
archive["EPUB/text/0001-文章.xhtml"],
|
||||
);
|
||||
assertStringIncludes(chapter, "../media/1/1.png");
|
||||
assertEquals(
|
||||
[...archive["EPUB/media/1/1.png"]],
|
||||
[...imageBytes],
|
||||
);
|
||||
} finally {
|
||||
await Deno.remove(root, { recursive: true });
|
||||
}
|
||||
});
|
||||
|
||||
Deno.test("packages shared assets directory images as EPUB resources", async () => {
|
||||
const root = await Deno.makeTempDir();
|
||||
const publication = `${root}/publication`;
|
||||
const templates = `${root}/templates`;
|
||||
const assets = `${root}/assets`;
|
||||
const output = `${root}/shared-assets.epub`;
|
||||
await Deno.mkdir(`${publication}/1::专栏`, { recursive: true });
|
||||
await Deno.mkdir(templates, { recursive: true });
|
||||
await Deno.mkdir(`${assets}/pictures`, { recursive: true });
|
||||
await Deno.writeTextFile(
|
||||
`${publication}/1::专栏/1::文章.md`,
|
||||
"",
|
||||
);
|
||||
const imageBytes = new Uint8Array([137, 80, 78, 71, 13, 10, 26, 10]);
|
||||
await Deno.writeFile(`${assets}/pictures/shared.png`, imageBytes);
|
||||
const builder = new EpubBuilder({
|
||||
asset_base: assets,
|
||||
journal: {
|
||||
build_mode: "detached",
|
||||
from_directory: publication,
|
||||
template_directory: templates,
|
||||
identifier: "urn:test:shared-assets",
|
||||
},
|
||||
epub: { path: output },
|
||||
}).with_files_sync();
|
||||
|
||||
try {
|
||||
await builder.build();
|
||||
const zip = inspectEpub(await Deno.readFile(output));
|
||||
const chapter = new TextDecoder().decode(
|
||||
zip["EPUB/text/0001-文章.xhtml"],
|
||||
);
|
||||
assertStringIncludes(chapter, "../media/1/1.png");
|
||||
assertEquals(zip["EPUB/media/1/1.png"], imageBytes);
|
||||
assertStringIncludes(
|
||||
new TextDecoder().decode(zip["EPUB/content.opf"]),
|
||||
'media/1/1.png" media-type="image/png"',
|
||||
);
|
||||
} finally {
|
||||
await Deno.remove(root, { recursive: true });
|
||||
}
|
||||
});
|
||||
@@ -30,7 +30,6 @@ import { assertEquals } from "@std/assert";
|
||||
// async fn() {
|
||||
// },
|
||||
// });
|
||||
import { ripgrep } from "ripgrep";
|
||||
Deno.test("default config is TOML with an asset base and vars table", async () => {
|
||||
const config = parseToml(await Deno.readTextFile("./config.toml")) as {
|
||||
asset_base: string;
|
||||
@@ -56,7 +55,7 @@ Deno.test("ripgrep test", async () => {
|
||||
await writer.write(
|
||||
new TextEncoder().encode("This is a test string with NULL in it."),
|
||||
);
|
||||
writer.releaseLock();
|
||||
await writer.close();
|
||||
const { code, stdout, stderr } = await child.output();
|
||||
console.log(`Exit code: ${code}`);
|
||||
console.log(`stdout: ${new TextDecoder().decode(stdout)}`);
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { assertEquals } from "@std/assert";
|
||||
import { DocuBuilder } from "./docu_builder.ts";
|
||||
import { md } from "../mdit/md.ts";
|
||||
|
||||
Deno.test("PDF rendering loads relative assets from configured file base", async () => {
|
||||
const chromiumPath = Deno.env.get("CHROMIUM_PATH");
|
||||
@@ -24,3 +25,65 @@ Deno.test("PDF rendering loads relative assets from configured file base", async
|
||||
assertEquals((await Deno.stat(pdfPath)).size > 0, true);
|
||||
await Deno.remove(pdfPath);
|
||||
});
|
||||
|
||||
Deno.test("PDF rendering waits for remote Markdown images", async () => {
|
||||
if (!Deno.env.get("CHROMIUM_PATH")) return;
|
||||
|
||||
const builder = new DocuBuilder({
|
||||
asset_base: "../../assets",
|
||||
journal: { build_js_script: false },
|
||||
});
|
||||
const pdfPath = await Deno.makeTempFile({ suffix: ".pdf" });
|
||||
await builder.with_browser();
|
||||
try {
|
||||
await builder.build_as_pdf({ path: pdfPath, format: "A4" }, [
|
||||
'<img src="https://imgbed.cmlmuf.org/i/2026/08/31/wd5mok.jpg" alt="Remote asset">',
|
||||
]);
|
||||
} finally {
|
||||
await builder.close_browser();
|
||||
}
|
||||
assertEquals((await Deno.stat(pdfPath)).size > 0, true);
|
||||
await Deno.remove(pdfPath);
|
||||
});
|
||||
|
||||
Deno.test("PDF rendering embeds publication-relative asset images", async () => {
|
||||
if (!Deno.env.get("CHROMIUM_PATH")) return;
|
||||
|
||||
const builder = new DocuBuilder({
|
||||
asset_base: "../../assets",
|
||||
journal: { build_js_script: false },
|
||||
});
|
||||
const resolvedHtml = (builder as any).resolveLocalAssetImages(
|
||||
'<img src="../../../assets/pictures/rsf.png">' +
|
||||
'<img src="https://example.test/remote.png">',
|
||||
);
|
||||
assertEquals(
|
||||
resolvedHtml.includes('src="data:image/png;base64,'),
|
||||
true,
|
||||
);
|
||||
assertEquals(
|
||||
resolvedHtml.includes('src="https://example.test/remote.png"'),
|
||||
true,
|
||||
);
|
||||
assertEquals(resolvedHtml.includes('loading="eager"'), false);
|
||||
const markdownHtml = md.render(
|
||||
"",
|
||||
);
|
||||
const eagerHtml = (builder as any).resolveLocalAssetImages(markdownHtml);
|
||||
assertEquals(eagerHtml.includes('loading="lazy"'), false);
|
||||
assertEquals(eagerHtml.includes('loading="eager"'), true);
|
||||
const pdfPath = await Deno.makeTempFile({ suffix: ".pdf" });
|
||||
await builder.with_browser();
|
||||
try {
|
||||
await builder.build_as_pdf({ path: pdfPath, format: "A4" }, [
|
||||
(builder as any).resolveLocalAssetImages(
|
||||
'<img src="../../../assets/pictures/26.11.0_pictures/ef1be4ca769fc52e513e69e1-129si8u.png" alt="Publication asset">',
|
||||
),
|
||||
]);
|
||||
} finally {
|
||||
await builder.close_browser();
|
||||
}
|
||||
const pdfText = new TextDecoder().decode(await Deno.readFile(pdfPath));
|
||||
assertEquals(pdfText.includes("/Subtype /Image"), true);
|
||||
await Deno.remove(pdfPath);
|
||||
});
|
||||
@@ -1,14 +1,15 @@
|
||||
import { Browser } from "playwright";
|
||||
import { resolve } from "node:path";
|
||||
import { pathToFileURL } from "node:url";
|
||||
import { PFiles } from "../file.ts";
|
||||
import { md } from "../mdit/md.ts";
|
||||
import { PFiles } from "../publication/file.ts";
|
||||
import { publicationToHtml } from "../document/inputs.ts";
|
||||
import type { HtmlDocument } from "../document/types.ts";
|
||||
import TurndownService from "turndown";
|
||||
|
||||
import "openpgp";
|
||||
|
||||
import { resolveChromiumPath } from "./utils.ts";
|
||||
import { renderTemplate, type TemplateVariables } from "../template_engine.ts";
|
||||
import { renderTemplate, type TemplateVariables } from "../template/engine.ts";
|
||||
|
||||
//原文->中间体(html)->目标格式(pdf/md/...)
|
||||
//生成pdf的流程:原文->中间体(html)->playwright渲染->pdf
|
||||
@@ -81,9 +82,54 @@ export class DocuBuilder {
|
||||
.trim();
|
||||
}
|
||||
|
||||
async as_markdown() {
|
||||
private resolveLocalAssetImages(html: string): string {
|
||||
const eagerHtml = html.replace(
|
||||
/(<img\b[^>]*\sloading=["'])lazy(["'])/gi,
|
||||
"$1eager$2",
|
||||
);
|
||||
return eagerHtml.replace(
|
||||
/(<img\b[^>]*\bsrc=["'])([^"']+)(["'])/gi,
|
||||
(_match, prefix: string, source: string, suffix: string) => {
|
||||
const assetMarker = source.match(/(?:^|\/)assets\/(.+)$/i);
|
||||
if (/^[a-z][a-z\d+.-]*:/i.test(source)) {
|
||||
return `${prefix}${source}${suffix}`;
|
||||
}
|
||||
try {
|
||||
const assetUrl = assetMarker
|
||||
? new URL(assetMarker[1], this.assetBaseHref)
|
||||
: new URL(source, this.assetBaseHref);
|
||||
const bytes = Deno.readFileSync(assetUrl);
|
||||
let binary = "";
|
||||
for (let offset = 0; offset < bytes.length; offset += 0x8000) {
|
||||
binary += String.fromCharCode(
|
||||
...bytes.subarray(offset, offset + 0x8000),
|
||||
);
|
||||
}
|
||||
const extension = assetUrl
|
||||
.pathname.split(".").at(-1)?.toLowerCase();
|
||||
const mime = extension === "png"
|
||||
? "image/png"
|
||||
: extension === "webp"
|
||||
? "image/webp"
|
||||
: extension === "gif"
|
||||
? "image/gif"
|
||||
: "image/jpeg";
|
||||
return `${prefix}data:${mime};base64,${btoa(binary)}${suffix}`;
|
||||
} catch (error) {
|
||||
console.warn(`Could not embed local image ${source}: ${error}`);
|
||||
return `${prefix}${
|
||||
assetMarker ? `${this.assetBaseHref}${assetMarker[1]}` : source
|
||||
}${suffix}`;
|
||||
}
|
||||
},
|
||||
);
|
||||
}
|
||||
|
||||
async as_markdown(document?: HtmlDocument) {
|
||||
const turndownService = new TurndownService();
|
||||
const html_page = await this.as_html_page();
|
||||
const html_page = await this.as_html_page(document, {
|
||||
articleMarkers: false,
|
||||
});
|
||||
const cleaned_html = html_page.replace(
|
||||
/<[^>]*class="[^"]*\bc_title\b[^"]*"[^>]*>([\s\S]*?)<\/[^>]+>/gi,
|
||||
"# $1",
|
||||
@@ -95,9 +141,11 @@ export class DocuBuilder {
|
||||
this.pages = [md_page];
|
||||
}
|
||||
|
||||
async as_txt(remove_links = true) {
|
||||
async as_txt(remove_links = true, document?: HtmlDocument) {
|
||||
const turndownService = new TurndownService();
|
||||
const html_page = await this.as_html_page();
|
||||
const html_page = await this.as_html_page(document, {
|
||||
articleMarkers: false,
|
||||
});
|
||||
const cleaned_html = html_page.replace(
|
||||
/<[^>]*class="[^"]*\bc_title\b[^"]*"[^>]*>([\s\S]*?)<\/[^>]+>/gi,
|
||||
"# $1",
|
||||
@@ -109,54 +157,56 @@ export class DocuBuilder {
|
||||
this.pages = [this.strip_markdown_symbols(md_page)];
|
||||
}
|
||||
|
||||
async as_html_page(): Promise<string> {
|
||||
async as_html_page(
|
||||
document?: HtmlDocument,
|
||||
options: { articleMarkers?: boolean } = {},
|
||||
): Promise<string> {
|
||||
this.pages = [];
|
||||
const htmlDocument = document ??
|
||||
(this.pfiles
|
||||
? publicationToHtml(this.pfiles, this.cfg, this.variables)
|
||||
: undefined);
|
||||
this.pages.push(htmlDocument?.layout.cover ?? "");
|
||||
this.pages.push(htmlDocument?.layout.styles ?? "");
|
||||
this.pages.push(htmlDocument?.layout.before ?? "");
|
||||
|
||||
this.pages.push(this.render(this.pfiles?.cover_html ?? ""));
|
||||
this.pages.push(this.render(this.pfiles?.style_html ?? ""));
|
||||
this.pages.push(this.render(this.pfiles?.other_htmls.join("\n") ?? ""));
|
||||
|
||||
if (this.pfiles != null) {
|
||||
let id = 1;
|
||||
this.pfiles.pcolumes.forEach((colume) => {
|
||||
let c_id = 1;
|
||||
this.pages.push(
|
||||
`<ch1 class="global c_title">${colume.name}</ch1>`,
|
||||
);
|
||||
colume.pfiles.forEach((pfile) => {
|
||||
const passageId = id++;
|
||||
const html = md.render(this.render(pfile.content, {
|
||||
document: {
|
||||
title: pfile.meta.title,
|
||||
index: pfile.meta.index,
|
||||
file: pfile.full_name,
|
||||
},
|
||||
column: {
|
||||
name: colume.name,
|
||||
index: colume.index,
|
||||
},
|
||||
}));
|
||||
let foundHeading = false;
|
||||
const indexedHtml = html.replace(
|
||||
/<h1(\b[^>]*)>([\s\S]*?)<\/h1>/i,
|
||||
(_match, attributes: string, title: string) => {
|
||||
foundHeading = true;
|
||||
return `<h1${attributes}><a href="https://wpep.invalid/article-start-${passageId}" style="color:inherit;text-decoration:none">${title}</a></h1>`;
|
||||
},
|
||||
);
|
||||
const markedHtml = foundHeading ? indexedHtml : indexedHtml.replace(
|
||||
/<p(\b[^>]*)>([\s\S]*?)<\/p>/i,
|
||||
(_match, attributes: string, content: string) =>
|
||||
`<p${attributes}><a href="https://wpep.invalid/article-start-${passageId}" style="color:inherit;text-decoration:none">${content}</a></p>`,
|
||||
);
|
||||
const printableHtml = markedHtml;
|
||||
if (htmlDocument) {
|
||||
let previousColumn: string | undefined;
|
||||
let c_id = 0;
|
||||
htmlDocument.chapters.forEach((chapter, index) => {
|
||||
const first = previousColumn !== chapter.column.name;
|
||||
if (first) {
|
||||
c_id = 0;
|
||||
this.pages.push(
|
||||
`<div class="global c${c_id++} passages" id="page-${passageId}">${printableHtml}</div>`,
|
||||
`<ch1 class="global c_title">${chapter.column.name}</ch1>`,
|
||||
);
|
||||
});
|
||||
}
|
||||
previousColumn = chapter.column.name;
|
||||
const passageId = index + 1;
|
||||
const firstPassageClass = first ? " first-passage" : "";
|
||||
const html = this.resolveLocalAssetImages(chapter.html);
|
||||
let foundHeading = false;
|
||||
const indexedHtml = html.replace(
|
||||
/<h1(\b[^>]*)>([\s\S]*?)<\/h1>/i,
|
||||
(_match, attributes: string, title: string) => {
|
||||
foundHeading = true;
|
||||
return `<h1${attributes}><a href="https://wpep.invalid/article-start-${passageId}" style="color:inherit;text-decoration:none">${title}</a></h1>`;
|
||||
},
|
||||
);
|
||||
const markedHtml = foundHeading ? indexedHtml : indexedHtml.replace(
|
||||
/<p(\b[^>]*)>([\s\S]*?)<\/p>/i,
|
||||
(_match, attributes: string, content: string) =>
|
||||
`<p${attributes}><a href="https://wpep.invalid/article-start-${passageId}" style="color:inherit;text-decoration:none">${content}</a></p>`,
|
||||
);
|
||||
const printableHtml = options.articleMarkers === false
|
||||
? html
|
||||
: markedHtml;
|
||||
this.pages.push(
|
||||
`<div class="global c${++c_id} passages${firstPassageClass}" id="page-${passageId}">${printableHtml}</div>`,
|
||||
);
|
||||
});
|
||||
}
|
||||
this.pages.push(this.render(this.pfiles?.end_html ?? ""));
|
||||
this.pages.push(htmlDocument?.layout.after ?? "");
|
||||
if (this.cfg.journal.build_js_script != false) {
|
||||
const f = Deno.readTextFileSync(`${this.cfg.journal.build_js_script}`);
|
||||
const html = this.pages.join("\n");
|
||||
@@ -266,7 +316,53 @@ return document?.documentElement?.outerHTML ?? html;`,
|
||||
const htmlPath = await Deno.makeTempFile({ suffix: ".html" });
|
||||
try {
|
||||
await Deno.writeTextFile(htmlPath, htmlWithBase);
|
||||
await page.goto(pathToFileURL(htmlPath).href, { timeout: 120000 });
|
||||
await page.goto(pathToFileURL(htmlPath).href, {
|
||||
timeout: 120000,
|
||||
waitUntil: "domcontentloaded",
|
||||
});
|
||||
|
||||
// `page.goto(..., { waitUntil: "load" })` does not guarantee that
|
||||
// every image has finished decoding before printing. Wait for each
|
||||
// image explicitly so remote Markdown images are included in the PDF.
|
||||
const failedImages = await page.evaluate(async () => {
|
||||
// The project is type-checked without the DOM library; these values
|
||||
// live in Chromium's page context, not in the Deno runtime.
|
||||
const pageDocument = (globalThis as any).document;
|
||||
const images = Array.from(pageDocument.images) as any[];
|
||||
const timedOutImages = await Promise.all(images.map((image) => {
|
||||
if (image.complete) return Promise.resolve(false);
|
||||
return new Promise<boolean>((resolve) => {
|
||||
let settled = false;
|
||||
const finish = () => {
|
||||
if (settled) return;
|
||||
settled = true;
|
||||
clearTimeout(timeout);
|
||||
resolve(false);
|
||||
};
|
||||
const timeout = setTimeout(() => {
|
||||
if (settled) return;
|
||||
settled = true;
|
||||
resolve(true);
|
||||
}, 15000);
|
||||
image.addEventListener("load", finish, { once: true });
|
||||
image.addEventListener("error", finish, { once: true });
|
||||
});
|
||||
}));
|
||||
const failed = images
|
||||
.filter((image) => !image.naturalWidth)
|
||||
.map((image) => image.currentSrc || image.src);
|
||||
return failed.length > 0
|
||||
? failed
|
||||
: images.filter((_image, index) => timedOutImages[index])
|
||||
.map((image) => image.currentSrc || image.src);
|
||||
});
|
||||
if (failedImages.length > 0) {
|
||||
console.warn(
|
||||
`Continuing PDF generation with ${failedImages.length} image(s) not loaded within 15 seconds: ${
|
||||
failedImages.join(", ")
|
||||
}`,
|
||||
);
|
||||
}
|
||||
|
||||
await page.pdf(pw_config);
|
||||
} finally {
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
import { PFiles } from "../file.ts";
|
||||
import { PFiles } from "../publication/file.ts";
|
||||
import { PDFParse } from "pdf-parse";
|
||||
import {
|
||||
PDFArray,
|
||||
@@ -7,9 +7,11 @@ import {
|
||||
PDFHexString,
|
||||
PDFName,
|
||||
PDFNumber,
|
||||
PDFString,
|
||||
} from "pdf-lib";
|
||||
import "openpgp";
|
||||
import { DocuBuilder } from "./docu_builder.ts";
|
||||
import type { HtmlDocument } from "../document/types.ts";
|
||||
|
||||
export function normalizePdfText(text: string): string {
|
||||
return text.normalize("NFKC").replace(
|
||||
@@ -95,7 +97,11 @@ export class IndexBuilder {
|
||||
passage_dir: string = "";
|
||||
file_path: string = "";
|
||||
cfg: any;
|
||||
constructor(file_path: string, cfg: any) {
|
||||
constructor(
|
||||
file_path: string,
|
||||
cfg: any,
|
||||
private readonly document?: HtmlDocument,
|
||||
) {
|
||||
if (cfg.journal.build_mode == "flat") {
|
||||
this.template_dir = cfg.journal.from_directory;
|
||||
this.passage_dir = cfg.journal.from_directory;
|
||||
@@ -112,6 +118,21 @@ export class IndexBuilder {
|
||||
passages_index: Map<string, number>;
|
||||
passages_column_name: Map<string, string>;
|
||||
} {
|
||||
if (this.document) {
|
||||
return {
|
||||
titles: this.document.chapters.map((chapter) => chapter.title),
|
||||
passages_index: new Map(
|
||||
this.document.chapters.map((
|
||||
chapter,
|
||||
) => [chapter.title, chapter.index]),
|
||||
),
|
||||
passages_column_name: new Map(
|
||||
this.document.chapters.map((
|
||||
chapter,
|
||||
) => [chapter.title, chapter.column.name]),
|
||||
),
|
||||
};
|
||||
}
|
||||
const pfiles = new PFiles(mode, this.passage_dir, this.cfg);
|
||||
const titles: Array<string> = [];
|
||||
const passages_index: Map<string, number> = new Map();
|
||||
@@ -127,7 +148,7 @@ export class IndexBuilder {
|
||||
}
|
||||
|
||||
async read_pdf() {
|
||||
const parser = new PDFParse({ url: this.file_path });
|
||||
const parser = new PDFParse({ data: await Deno.readFile(this.file_path) });
|
||||
try {
|
||||
const pdfText = await parser.getText();
|
||||
this.total_pages = pdfText.total;
|
||||
@@ -136,30 +157,38 @@ export class IndexBuilder {
|
||||
await parser.destroy();
|
||||
}
|
||||
}
|
||||
//修改实现为在每页文本中查找目录标题,支持目录标题跨行的情况。通过维护一个标题游标,逐页扫描PDF文本,尝试匹配当前标题。每页文本被拆分成行,并进行规范化(去除空格和换行)。对于每个标题,尝试将连续的行合并起来进行匹配,如果找到匹配则记录该标题对应的页码和章节索引,并移动到下一个标题。这样可以更准确地处理目录标题在PDF中被拆分成多行的情况。
|
||||
async build_index() {
|
||||
const pdf_obj = await this.read_pdf();
|
||||
this.indexed_map.clear();
|
||||
const { titles, passages_index, passages_column_name } = this
|
||||
.get_index_titles_from_file_name(
|
||||
this.cfg.journal.build_mode ?? "flat",
|
||||
);
|
||||
// Normalize titles (remove spaces/newlines) and require exact equality.
|
||||
// Matching supports title text split across multiple PDF lines.
|
||||
const pageTexts = Array.from(
|
||||
{ length: this.total_pages },
|
||||
(_, index) => pdf_obj.getPageText(index + 1),
|
||||
);
|
||||
const searchOptions = (this.cfg.indexSearch ?? {}) as IndexSearchOptions;
|
||||
const matchedPages = findTitlePages(
|
||||
titles,
|
||||
pageTexts,
|
||||
searchOptions,
|
||||
);
|
||||
|
||||
const markerPages = await this.getArticleStartPages();
|
||||
const missingIndices = titles.map((_, index) => index).filter((index) =>
|
||||
!markerPages.has(index + 1)
|
||||
);
|
||||
const fallbackPages = new Map<number, number | undefined>();
|
||||
if (missingIndices.length > 0) {
|
||||
console.warn(
|
||||
`Missing article markers (${missingIndices.length}/${titles.length}); falling back to title search`,
|
||||
);
|
||||
const pdf = await this.read_pdf();
|
||||
const pageTexts = Array.from(
|
||||
{ length: this.total_pages },
|
||||
(_, index) => pdf.getPageText(index + 1),
|
||||
);
|
||||
const matchedPages = findTitlePages(
|
||||
missingIndices.map((index) => titles[index]),
|
||||
pageTexts,
|
||||
(this.cfg.indexSearch ?? {}) as IndexSearchOptions,
|
||||
);
|
||||
missingIndices.forEach((index, offset) =>
|
||||
fallbackPages.set(index, matchedPages[offset])
|
||||
);
|
||||
}
|
||||
|
||||
for (const [index, title] of titles.entries()) {
|
||||
const page = markerPages.get(index + 1) ?? matchedPages[index];
|
||||
const page = markerPages.get(index + 1) ?? fallbackPages.get(index);
|
||||
if (page !== undefined && !this.indexed_map.has(title)) {
|
||||
this.indexed_map.set(title, {
|
||||
page,
|
||||
@@ -168,10 +197,14 @@ export class IndexBuilder {
|
||||
});
|
||||
}
|
||||
}
|
||||
const missingTitles = titles.filter((title) => !this.indexed_map.has(title));
|
||||
const missingTitles = titles.filter((title) =>
|
||||
!this.indexed_map.has(title)
|
||||
);
|
||||
if (missingTitles.length > 0) {
|
||||
throw new Error(
|
||||
`Index coverage incomplete (${this.indexed_map.size}/${titles.length}); missing: ${missingTitles.join("、")}`,
|
||||
`Index coverage incomplete (${this.indexed_map.size}/${titles.length}); missing: ${
|
||||
missingTitles.join("、")
|
||||
}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
@@ -179,6 +212,7 @@ export class IndexBuilder {
|
||||
private async getArticleStartPages(): Promise<Map<number, number>> {
|
||||
const parser = await PDFDocument.load(await Deno.readFile(this.file_path));
|
||||
const pages = parser.getPages();
|
||||
this.total_pages = pages.length;
|
||||
const markerPages = new Map<number, number>();
|
||||
const markerPattern = /^https:\/\/wpep\.invalid\/article-start-(\d+)$/;
|
||||
|
||||
@@ -194,8 +228,8 @@ export class IndexBuilder {
|
||||
const actionValue = annotation.lookup(PDFName.of("A"));
|
||||
const action = actionValue instanceof PDFDict ? actionValue : undefined;
|
||||
const uri = action?.lookup(PDFName.of("URI"));
|
||||
const url = uri && "decodeText" in uri
|
||||
? (uri as PDFHexString).decodeText()
|
||||
const url = uri instanceof PDFString || uri instanceof PDFHexString
|
||||
? uri.decodeText()
|
||||
: "";
|
||||
const match = markerPattern.exec(url);
|
||||
if (match) {
|
||||
@@ -358,10 +392,14 @@ export class IndexBuilder {
|
||||
const actionValue = annotation.lookup(PDFName.of("A"));
|
||||
const action = actionValue instanceof PDFDict ? actionValue : undefined;
|
||||
const uri = action?.lookup(PDFName.of("URI"));
|
||||
const url = uri && "decodeText" in uri
|
||||
? (uri as PDFHexString).decodeText()
|
||||
const url = uri instanceof PDFString || uri instanceof PDFHexString
|
||||
? uri.decodeText()
|
||||
: "";
|
||||
return !markerPattern.test(url);
|
||||
if (!markerPattern.test(url)) return true;
|
||||
// Scrub the action too: detached annotation objects may still be saved.
|
||||
action?.delete(PDFName.of("URI"));
|
||||
annotation.delete(PDFName.of("A"));
|
||||
return false;
|
||||
});
|
||||
page.node.set(PDFName.of("Annots"), pdfDoc.context.obj(retainedRefs));
|
||||
}
|
||||
@@ -381,7 +419,7 @@ export class IndexBuilder {
|
||||
}
|
||||
const sourcePages = sourcePdf.getPages();
|
||||
let annotationsAdded = 0;
|
||||
let entryIndex = 0;
|
||||
const linkedPages = new Set<number>();
|
||||
sourcePages.forEach((sourcePage, pageIndex) => {
|
||||
const sourceAnnots = sourcePage.node.Annots();
|
||||
if (!sourceAnnots) return;
|
||||
@@ -395,12 +433,13 @@ export class IndexBuilder {
|
||||
const uriText = uri && "decodeText" in uri
|
||||
? (uri as any).decodeText()
|
||||
: "";
|
||||
if (!/^https:\/\/wpep\.invalid\/index-page-\d+$/.test(uriText)) {
|
||||
const match = /^https:\/\/wpep\.invalid\/index-page-(\d+)$/.exec(
|
||||
uriText,
|
||||
);
|
||||
if (!match) {
|
||||
continue;
|
||||
}
|
||||
const entry = sortedEntries[entryIndex++];
|
||||
if (!entry) continue;
|
||||
const pageNumber = entry[1].page;
|
||||
const pageNumber = Number(match[1]);
|
||||
const sourceRect = sourceAnnot.lookup(PDFName.of("Rect"), PDFArray);
|
||||
if (!sourceRect || sourceRect.size() !== 4) continue;
|
||||
const mappedPage = this.remap_page_after_index_insert(
|
||||
@@ -417,7 +456,15 @@ export class IndexBuilder {
|
||||
];
|
||||
const targetIndexPage = indexPages[pageIndex];
|
||||
if (!targetIndexPage) continue;
|
||||
sourceAnnot.set(
|
||||
const targetAnnot = ctx.obj({
|
||||
Type: "Annot",
|
||||
Subtype: "Link",
|
||||
Rect: sourceRect.asArray().map((value) =>
|
||||
sourcePdf.context.lookup(value, PDFNumber).asNumber()
|
||||
),
|
||||
Border: [0, 0, 0],
|
||||
});
|
||||
targetAnnot.set(
|
||||
PDFName.of("Dest"),
|
||||
ctx.obj([
|
||||
(destinationPage as any).ref,
|
||||
@@ -425,15 +472,15 @@ export class IndexBuilder {
|
||||
PDFNumber.of(destinationPage.getSize().height),
|
||||
]),
|
||||
);
|
||||
sourceAnnot.delete(PDFName.of("A"));
|
||||
const ref = ctx.register(sourceAnnot);
|
||||
const ref = ctx.register(targetAnnot);
|
||||
const annots = targetIndexPage.node.Annots();
|
||||
if (annots) annots.push(ref);
|
||||
else targetIndexPage.node.set(PDFName.of("Annots"), ctx.obj([ref]));
|
||||
annotationsAdded++;
|
||||
linkedPages.add(pageNumber);
|
||||
}
|
||||
});
|
||||
if (annotationsAdded !== sortedEntries.length) {
|
||||
if (sortedEntries.some(([, entry]) => !linkedPages.has(entry.page))) {
|
||||
throw new Error(
|
||||
`Chromium index PDF link annotation mismatch: expected ${sortedEntries.length}, found ${annotationsAdded}`,
|
||||
);
|
||||
@@ -506,8 +553,7 @@ export class IndexBuilder {
|
||||
}
|
||||
});
|
||||
|
||||
// deno-lint-ignore no-explicit-any
|
||||
const catalog = (pdfDoc.catalog as any).dict;
|
||||
const catalog = pdfDoc.catalog;
|
||||
catalog.set(PDFName.of("Outlines"), outlinesRef);
|
||||
if (open_outline_by_default) {
|
||||
catalog.set(PDFName.of("PageMode"), PDFName.of("UseOutlines"));
|
||||
|
||||
@@ -2,8 +2,153 @@ import { assertEquals } from "@std/assert";
|
||||
import {
|
||||
findTitlePage,
|
||||
findTitlePages,
|
||||
IndexBuilder,
|
||||
normalizePdfText,
|
||||
} from "./index_builder.ts";
|
||||
import { PDFArray, PDFDict, PDFDocument, PDFName, PDFString } from "pdf-lib";
|
||||
|
||||
function addLink(pdf: PDFDocument, pageIndex: number, url: string) {
|
||||
const annotation = pdf.context.obj({
|
||||
Type: "Annot",
|
||||
Subtype: "Link",
|
||||
Rect: [10, 10, 100, 30],
|
||||
A: { S: "URI", URI: PDFString.of(url) },
|
||||
});
|
||||
pdf.getPage(pageIndex).node.addAnnot(pdf.context.register(annotation));
|
||||
}
|
||||
|
||||
function markerBuilder(path: string) {
|
||||
const builder = new IndexBuilder(path, {
|
||||
journal: { build_mode: "flat", from_directory: "." },
|
||||
});
|
||||
builder.get_index_titles_from_file_name = () => ({
|
||||
titles: ["Article A", "Article B"],
|
||||
passages_index: new Map([["Article A", 1], ["Article B", 2]]),
|
||||
passages_column_name: new Map([["Article A", "Column"], [
|
||||
"Article B",
|
||||
"Column",
|
||||
]]),
|
||||
});
|
||||
return builder;
|
||||
}
|
||||
|
||||
Deno.test("marker index skips text extraction and uses earliest wrapped marker", async () => {
|
||||
const path = await Deno.makeTempFile({ suffix: ".pdf" });
|
||||
try {
|
||||
const pdf = await PDFDocument.create();
|
||||
for (let i = 0; i < 4; i++) pdf.addPage();
|
||||
addLink(pdf, 1, "https://wpep.invalid/article-start-1");
|
||||
addLink(pdf, 2, "https://wpep.invalid/article-start-1");
|
||||
addLink(pdf, 2, "https://wpep.invalid/article-start-2");
|
||||
await Deno.writeFile(path, await pdf.save());
|
||||
const builder = markerBuilder(path);
|
||||
builder.read_pdf = () => {
|
||||
throw new Error("Text extraction must not run");
|
||||
};
|
||||
await builder.build_index();
|
||||
assertEquals(builder.total_pages, 4);
|
||||
assertEquals([...builder.indexed_map.values()].map((entry) => entry.page), [
|
||||
2,
|
||||
3,
|
||||
]);
|
||||
} finally {
|
||||
await Deno.remove(path);
|
||||
}
|
||||
});
|
||||
|
||||
Deno.test("missing markers fall back to text without replacing marker matches", async () => {
|
||||
const path = await Deno.makeTempFile({ suffix: ".pdf" });
|
||||
try {
|
||||
const pdf = await PDFDocument.create();
|
||||
pdf.addPage();
|
||||
pdf.addPage();
|
||||
pdf.getPage(0).drawText("Article A Article B");
|
||||
addLink(pdf, 1, "https://wpep.invalid/article-start-1");
|
||||
await Deno.writeFile(path, await pdf.save());
|
||||
const builder = markerBuilder(path);
|
||||
await builder.build_index();
|
||||
assertEquals([...builder.indexed_map.values()].map((entry) => entry.page), [
|
||||
2,
|
||||
1,
|
||||
]);
|
||||
} finally {
|
||||
await Deno.remove(path);
|
||||
}
|
||||
});
|
||||
|
||||
Deno.test("merge removes marker URLs, preserves reader links and retargets wrapped TOC and outlines", async () => {
|
||||
const dir = await Deno.makeTempDir();
|
||||
const path = `${dir}/main.pdf`;
|
||||
const indexPath = `${dir}/index.pdf`;
|
||||
try {
|
||||
const pdf = await PDFDocument.create();
|
||||
for (let i = 0; i < 4; i++) pdf.addPage();
|
||||
addLink(pdf, 1, "https://wpep.invalid/article-start-1");
|
||||
addLink(pdf, 2, "https://wpep.invalid/article-start-2");
|
||||
addLink(pdf, 1, "https://example.com/reader");
|
||||
pdf.getPage(1).drawText("Article A");
|
||||
await Deno.writeFile(path, await pdf.save());
|
||||
const index = await PDFDocument.create();
|
||||
index.addPage();
|
||||
addLink(index, 0, "https://wpep.invalid/index-page-2");
|
||||
addLink(index, 0, "https://wpep.invalid/index-page-2");
|
||||
addLink(index, 0, "https://wpep.invalid/index-page-3");
|
||||
await Deno.writeFile(indexPath, await index.save());
|
||||
const builder = markerBuilder(path);
|
||||
await builder.build_index();
|
||||
const inserted = await builder.combine_index_pdf_to_main_pdf(indexPath, 2);
|
||||
await builder.add_pdf_outline({
|
||||
inserted_index_pages: inserted,
|
||||
insert_index_after_page: 2,
|
||||
});
|
||||
const result = await PDFDocument.load(await Deno.readFile(path));
|
||||
assertEquals(result.getPageCount(), 5);
|
||||
const links = result.getPage(2).node.Annots()!;
|
||||
assertEquals(links.size(), 3);
|
||||
const destinations = links.asArray().map((ref) => {
|
||||
const annotation = result.context.lookup(ref, PDFDict);
|
||||
return annotation.lookup(PDFName.of("Dest"), PDFArray).get(0).toString();
|
||||
});
|
||||
assertEquals(destinations, [
|
||||
result.getPage(1).ref.toString(),
|
||||
result.getPage(1).ref.toString(),
|
||||
result.getPage(3).ref.toString(),
|
||||
]);
|
||||
const readerLinks = result.getPage(1).node.Annots()!;
|
||||
assertEquals(readerLinks.size(), 1);
|
||||
const reader = result.context.lookup(readerLinks.get(0), PDFDict);
|
||||
assertEquals(
|
||||
reader.lookup(PDFName.of("A"), PDFDict).lookup(
|
||||
PDFName.of("URI"),
|
||||
PDFString,
|
||||
).decodeText(),
|
||||
"https://example.com/reader",
|
||||
);
|
||||
const outlines = result.catalog.lookup(PDFName.of("Outlines"), PDFDict);
|
||||
const first = outlines.lookup(PDFName.of("First"), PDFDict);
|
||||
const second = first.lookup(PDFName.of("Next"), PDFDict);
|
||||
assertEquals(
|
||||
first.lookup(PDFName.of("Dest"), PDFArray).get(0).toString(),
|
||||
result.getPage(1).ref.toString(),
|
||||
);
|
||||
assertEquals(
|
||||
second.lookup(PDFName.of("Dest"), PDFArray).get(0).toString(),
|
||||
result.getPage(3).ref.toString(),
|
||||
);
|
||||
for (const [, object] of result.context.enumerateIndirectObjects()) {
|
||||
if (object instanceof PDFDict) {
|
||||
const uri = object.lookup(PDFName.of("URI"));
|
||||
if (uri instanceof PDFString) {
|
||||
assertEquals(uri.decodeText().includes("article-start-"), false);
|
||||
}
|
||||
}
|
||||
}
|
||||
const cleanedText = await builder.read_pdf();
|
||||
assertEquals(cleanedText.getPageText(2).includes("Article A"), true);
|
||||
} finally {
|
||||
await Deno.remove(dir, { recursive: true });
|
||||
}
|
||||
});
|
||||
|
||||
Deno.test("normalizes title whitespace and line breaks", () => {
|
||||
assertEquals(normalizePdfText("工人\n 解放 报"), "工人解放报");
|
||||
|
||||
@@ -1,16 +1,11 @@
|
||||
import { OpenPGPSigner } from "./pdf_builder/utils.ts";
|
||||
import { DocuBuilder } from "./pdf_builder/docu_builder.ts";
|
||||
import { IndexBuilder } from "./pdf_builder/index_builder.ts";
|
||||
import type { TemplateVariables } from "./template_engine.ts";
|
||||
import { OpenPGPSigner } from "./utils.ts";
|
||||
import { DocuBuilder } from "./docu_builder.ts";
|
||||
import { IndexBuilder } from "./index_builder.ts";
|
||||
import type { TemplateVariables } from "../template/engine.ts";
|
||||
import { PublicationInput } from "../document/inputs.ts";
|
||||
import type { DocumentConfig, HtmlDocument } from "../document/types.ts";
|
||||
|
||||
export type GPConfig = {
|
||||
journal: {
|
||||
from_directory: string;
|
||||
template_directory?: string;
|
||||
build_mode?: "detached" | "flat";
|
||||
build_js_script?: string | false;
|
||||
};
|
||||
asset_base?: string;
|
||||
export type GPConfig = DocumentConfig & {
|
||||
pdf: {
|
||||
generate_index?: boolean;
|
||||
use_qpdf?: {
|
||||
@@ -42,14 +37,19 @@ export type PDFPipelineOptions = {
|
||||
export async function processPdfConfig(
|
||||
config: GPConfig,
|
||||
options: PDFPipelineOptions = {},
|
||||
document?: HtmlDocument,
|
||||
) {
|
||||
const builder = new DocuBuilder(config, options.variables);
|
||||
const input = document ??
|
||||
await new PublicationInput(config, options.variables).read();
|
||||
const builder = new DocuBuilder(
|
||||
{ ...config, asset_base: input.assetBase },
|
||||
options.variables,
|
||||
);
|
||||
|
||||
try {
|
||||
await builder.with_browser();
|
||||
console.debug(`Processing config: ${JSON.stringify(config)}`);
|
||||
builder.with_files_sync(config.journal.from_directory);
|
||||
await builder.as_html_page();
|
||||
await builder.as_html_page(input);
|
||||
await builder.build_as_pdf(config.playwright);
|
||||
} finally {
|
||||
await builder.close_browser();
|
||||
@@ -64,6 +64,7 @@ export async function processPdfConfig(
|
||||
config,
|
||||
insertIndexAfterPage,
|
||||
options.keepIndexPdf ?? false,
|
||||
input,
|
||||
);
|
||||
}
|
||||
|
||||
@@ -80,10 +81,15 @@ async function addIndexToPdf(
|
||||
config: GPConfig,
|
||||
insertIndexAfterPage = 1,
|
||||
keepIndexPdf = false,
|
||||
document?: HtmlDocument,
|
||||
) {
|
||||
console.log(`Adding index to PDF ${config.playwright.path}...`);
|
||||
|
||||
const indexBuilder = new IndexBuilder(config.playwright.path, config);
|
||||
const indexBuilder = new IndexBuilder(
|
||||
config.playwright.path,
|
||||
config,
|
||||
document,
|
||||
);
|
||||
await indexBuilder.build_index();
|
||||
console.log(
|
||||
`Index entries for ${config.playwright.path}: ${indexBuilder.indexed_map.size}`,
|
||||
@@ -121,6 +127,7 @@ async function addIndexToPdf(
|
||||
}
|
||||
await indexBuilder.add_pdf_outline({
|
||||
inserted_index_pages: insertedIndexPages,
|
||||
insert_index_after_page: insertIndexAfterPage,
|
||||
});
|
||||
|
||||
if (keepIndexPdf) {
|
||||
@@ -1,5 +1,8 @@
|
||||
import { type GPConfig, processPdfConfig } from "./pdf_pipeline.ts";
|
||||
import type { TemplateVariables } from "./template_engine.ts";
|
||||
import { type GPConfig } from "./pipeline.ts";
|
||||
import { PublicationInput } from "../document/inputs.ts";
|
||||
import { PdfOutput } from "../document/outputs.ts";
|
||||
import { convertDocument } from "../document/types.ts";
|
||||
import type { TemplateVariables } from "../template/engine.ts";
|
||||
|
||||
type WorkerRequest = {
|
||||
id: number;
|
||||
@@ -26,7 +29,10 @@ const workerGlobal = self as unknown as {
|
||||
workerGlobal.onmessage = async (event: MessageEvent<WorkerRequest>) => {
|
||||
const { id, config, options } = event.data;
|
||||
try {
|
||||
await processPdfConfig(config, options);
|
||||
await convertDocument(
|
||||
new PublicationInput(config, options.variables),
|
||||
new PdfOutput(config, options),
|
||||
);
|
||||
workerGlobal.postMessage({ id, ok: true } satisfies WorkerResponse);
|
||||
} catch (error) {
|
||||
workerGlobal.postMessage(
|
||||
@@ -33,12 +33,14 @@ Deno.test("parses detached column folders without an order prefix", () => {
|
||||
});
|
||||
|
||||
Deno.test("parses finite JavaScript numbers as detached column order", () => {
|
||||
for (const [prefix, expected] of [
|
||||
["-2.5", -2.5],
|
||||
["+3.25", 3.25],
|
||||
["1e3", 1000],
|
||||
["0x10", 16],
|
||||
]) {
|
||||
for (
|
||||
const [prefix, expected] of [
|
||||
["-2.5", -2.5],
|
||||
["+3.25", 3.25],
|
||||
["1e3", 1000],
|
||||
["0x10", 16],
|
||||
]
|
||||
) {
|
||||
const meta = new PFileMetadata(`/publication/${prefix}::专栏/1::文章.md`);
|
||||
assertEquals(meta.colume, "专栏");
|
||||
assertEquals(meta.c_index, expected);
|
||||
@@ -1,5 +1,5 @@
|
||||
import { basename, dirname, join, relative, resolve } from "node:path";
|
||||
import { cfg } from "./main.ts";
|
||||
import { cfg } from "../main.ts";
|
||||
|
||||
type WebDavOptions = Record<string, unknown>;
|
||||
type WebDavConfig = {
|
||||
@@ -1,6 +1,6 @@
|
||||
import { assertEquals } from "@std/assert";
|
||||
import { parse as parseToml } from "@std/toml";
|
||||
import { createTemplateVariables, renderTemplate } from "./template_engine.ts";
|
||||
import { createTemplateVariables, renderTemplate } from "./engine.ts";
|
||||
|
||||
Deno.test("renders Nunjucks variables, conditionals, loops, and constants", () => {
|
||||
const result = renderTemplate(
|
||||