DenysMedvid/pdf2htmlEX
1
0
Fork 0
mirror of https://github.com/pdf2htmlEX/pdf2htmlEX.git synced 2024-12-21 20:50:07 +00:00
Code

Merge pull request #29 from stephengaito/updateBuildScripts

Browse Source 
Update build scripts

Major refactor of build scripts to ensure we can build pdf2htmlEX with staticly linked versions of poppler and fontforge.

Added an "experimental" build based on Alpine to produce an Alpine based docker image. NOTE: the Alpine iconv is not sufficient for all fonts required for pdf2htmlEX/poppler. This image may or may not work for your PDFs.

Fully tested using the associated Laminar scripts in stephengaito/laminar-scripts and stephengaito/pdf2htmlex-laminar-scripts
This commit is contained in:
stephengaito 2020-06-22 10:26:39 +01:00 committed by GitHub
commit 9a988599ef
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
70 changed files with 1349 additions and 437 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

26
.travis.yml
View File

@ -17,26 +17,12 @@ branches:
jobs:
fast_finish: true
include:
- stage: build-linux
os: linux
# env:
# global: DOCKER_FROM="ubuntu:18.04"
# services:
# - docker
- os: linux
dist: bionic
env: DOCKER_FROM="ubuntu:18.04"
services:
- docker
addons:
firefox: latest
script:
- ./buildScripts/travisLinuxDoItAll
# - stage: test
# os:
# - linux
# script: ./testScripts/runLocalTests
#deploy:
# provider: releases
# api_key:
# secure: ALWKAkgeLSPS4ULyTtxU50ZmSICEafCirCQ1YmuVTqe9nrc8E9Jm+4QaBlab5k6v90EpzNvZBnMOiT8cT1Wp2lqJzcGdGMcG49mhRvirdYhXgWuHYnygVgkxEhgejXcQaEiaGPDV3Y1k9MuiAVr8ApjUvIMlY4Hg1y0r7/rqR2fYmen+f+6a87bp5PptwakC9bfoYb92651d0TbugFfKT6TGKlE2jGlQ6On1alzLQ2DXA8coUjwpp4NcDD8OhnYGN/VZWkqckaBrMrHwjJ5FTfy5MGBOqCLzNix8sCGLEeCKkKC8J0nniNJ7+KOu9LJ2uIezKYfaUOJV0kik7F9ErvxjbfSRieako2yaSOR8X39aicTtpbyoXPRSKDU4WYX56vt4Dx7aNrYtP5b/UwWGXZa9MK7rNGJ+hiMDO1AzuyXirzR4RaJXCyC8n5W9ahBZBHCMqd5gMg5sMGPNj3Rgt34+R+nGYssum5wvr+6IKNl57sIoZFCgqviBrw2gJOCqlgOcEk/4ZSehWnXPCk6Gzxvkqkt/dmp8pJLNyvMaWcEZ6CXNHDU8MZS7bASo8Z7eAYvmX0MMjVty8J47expVaJ8M3OFHu6lE6tU8wwfd+Igr2TlWcixqaa3hDMgS+ghwdiZbYuMdR1AyZnEqK3xwvXXPPXrqRsB9xHtczLibkjY=
# file: pdf2htmlEX-newBuildSystem-x86_64.AppImage
# skip_cleanup: true
# on:
# tags: true
# repo: stephengaito/pdf2htmlEX

10
CONTRIBUTING.md
View File

@ -23,7 +23,7 @@ A few channels are available to reach the developers, please find the most prope
### The Issue Tracker
The [Issue Tracker](https://github.com/coolwanglu/pdf2htmlEX/issues)
The [Issue Tracker](https://github.com/pdf2htmlEX/pdf2htmlEX/issues)
is the best way for
[bug reports](#bug-reports),
[features requests](#feature-requests)
@ -54,7 +54,7 @@ However do not report issues or submit patches there, since it's terrible to kee
pdf2htmlEX is mostly written and maintained by 王璐 (Lu Wang).
His email and twitter account can be found in
[README.md](https://github.com/coolwanglu/pdf2htmlEX/blob/master/README.md).
[README.md](https://github.com/pdf2htmlEX/pdf2htmlEX/blob/master/README.md).
Please post only messages that do not fit into the above channels, otherwise
note that he no longer replies with _Please post your question to the mailing list_ or _Please file an issue at GitHub_, consider your message already replied.
@ -73,7 +73,7 @@ follow the following steps to get it resolved as fast as possible:
First of all, did you realize that your question might have been already answered in one of the following places?
- [pdf2htmlEX Wiki](https://github.com/coolwanglu/pdf2htmlEX/wiki)
- [pdf2htmlEX Wiki](https://github.com/pdf2htmlEX/pdf2htmlEX/wiki)
- The manpage (run `man pdf2htmlEX`)
- Old posts in the [mailing list](#the-mailing-list) or the [issue tracker](#the-issue-tracker)
- [Google](http://www.google.com/)
@ -93,7 +93,7 @@ Bugs should always be reported to [the Issue Tracker](#the-issue-tracker).
Before you report any bug:
- Use the latest git version of pdf2htmlEX, since the issue may have been already fixed.
- Search for previous issues (open or closed), to make sure that the issue has not been reported before.
- If pdf2htmlEX crashed, take a look at [this article](https://github.com/coolwanglu/pdf2htmlEX/wiki/Troubleshooting-Crashes).
- If pdf2htmlEX crashed, take a look at [this article](https://github.com/pdf2htmlEX/pdf2htmlEX/wiki/Troubleshooting-Crashes).
A good bug report shouldn't leave others needing to chase you up for more information.
The developers may be very familiar with the code base of pdf2htmlEX,
@ -110,7 +110,7 @@ If you are not sure, please try to answer the following questions:
- What steps will reproduce the issue? — please try to remove unnecessary steps
- What's the result and what did you expect? — e.g. you can post screenshots
- What error messages did you see?
- What's the affected PDF file and which pages are causing the issue? Create a pull request on [this repo](https://github.com/coolwanglu/pdf2htmlEX-testcase).
- What's the affected PDF file and which pages are causing the issue? Create a pull request on [this repo](https://github.com/pdf2htmlEX/pdf2htmlEX-testcase).
Especially for issues regarding building pdf2htmlEX:
- Which compiler are you using?

2
INSTALL
View File

@ -1,3 +1,3 @@
For instructions of building the source code, visit:
https://github.com/coolwanglu/pdf2htmlEX/wiki/Building
https://github.com/pdf2htmlEX/pdf2htmlEX/wiki/Building

2
LICENSE
View File

@ -1,4 +1,4 @@
pdf2htmlEX (https://github.com/coolwanglu/pdf2htmlEX)
pdf2htmlEX (https://github.com/pdf2htmlEX/pdf2htmlEX)
Copyright (c) 2012-2014 Lu Wang <coolwanglu@gmail.com> and other contributors
pdf2htmlEX, as a whole package, is licensed under GPLv3 (or any later

0
debian/changelog → archive/debian/changelog
View File

0
debian/compat → archive/debian/compat
View File

0
debian/control → archive/debian/control
View File

0
debian/copyright → archive/debian/copyright
View File

0
debian/dirs → archive/debian/dirs
View File

0
debian/pdf2htmlex.NEWS → archive/debian/pdf2htmlex.NEWS
View File

0
debian/pdf2htmlex.README → archive/debian/pdf2htmlex.README
View File

0
debian/pdf2htmlex.TODO → archive/debian/pdf2htmlex.TODO
View File

0
debian/rules → archive/debian/rules
View File

0
debian/source/format → archive/debian/source/format
View File

334
buildScripts/Readme.md Normal file
View File

@ -0,0 +1,334 @@
# Building pdf2htmlEX
Because of its intimate use of *specific* versions of both Poppler and
FontForge, cleanly building `pdf2htmlEX` is rather more complex than
normal.
The (shell) scripts in this directory help automate this mutli-stage
process.
For all but the most experienced programmers, we *strongly* encourage you
to use these scripts to build `pdf2htmlEX`.
---
**Table of contents**
- [TL;DR ...](#tldr-)
- [Downloading precompiled versions](#downloading-precompiled-versions-downloads)
- [Building yourself](#building-yourself)
- [The problem](#the-problem)
- [Our solution](#our-solution)
- [The gory details ...](#the-gory-details-)
- [Top-level scripts](#top-level-scripts)
- [Individual steps](#individual-steps)
- [Helper files and scripts](#helper-files-and-scripts)
- [Yet more details?](#yet-more-details)
---
## TL;DR ...
### Downloading precompiled versions
For most users, you probably really want to simply download one of the
[precompiled versions of
`pdf2htmlEX`](https://github.com/pdf2htmlEX/pdf2htmlEX/releases):
- [Debian archive](https://en.wikipedia.org/wiki/Dpkg) : Download,
[apt](https://en.wikipedia.org/wiki/APT_(software)) install locally,
and run...
This will work on any [Debian](https://www.debian.org/) based and most
recent Windows 10 machines.
Experienced users of Linux, may be able to repackage the `*.deb` we
provide for use with their favourite package management tool.
- [AppImage](https://appimage.org/) : Download, make executable, and
run...
This will work on most Linuxes, and most recent Windows 10.
(It will not currently work on MacOS or Alpine based machines).
- [Docker](https://www.docker.com/) Image from the [`pdf2htmlEX` Docker
hub](https://hub.docker.com/orgs/pdf2htmlex/repositories).
This will work on any machine with Docker installed.
(Note: that *advanced* use of `pdf2htmlEX` requires careful attention to
the configuration of various tools, such as fontconfig, iconv and your
locally available fonts use by the poppler and fontforge libraries. The
docker images created by the pdf2htmlEX team might not be as well
configured for *your needs* as a docker created and configured by you)
### Building yourself
To build `pdf2htmlEX` on a Debian/Apt related machine, inside the root
directory of a fresh clone of the
[pdf2htmlEX/pdf2htmlEX](https://github.com/pdf2htmlEX/pdf2htmlEX)
repository, type:
```
./buildScripts/buildInstallLocallyApt
```
This will automatically install all required development tools and
libraries, and then proceed to download and statically compile the
required versions of both Poppler and FontForge before compiling and
installing `pdf2htmlEX` into /usr/local/bin.
**NOTE:** at the moment this will **only** work on machines with a
[Debian](https://www.debian.org/) based distribution. such as
[Ubuntu](https://ubuntu.com/), [Linux Mint](https://linuxmint.com/), etc.
**NOTE:** there is currently an *experimental* build script,
`./buildScripts/buildInstallLocallyAlpine`, for builds in Alpine
environments.
## The problem
To provide its full functionality, the `pdf2htmlEX` sources make direct
use of source code and unexposed methods from both the Poppler and
FontForge projects. Unfortunately the source code in the Poppler and
FontForge projects that the `pdf2htmlEX` uses changes regularly.
This means that the `pdf2htmlEX` souce code *must* be updated regularly to
match *specific releases* of both Poppler and FontForge.
Unfortunately, the installed versions of both Poppler and FontForge in
most Linux distributions, lag the official releases of both of these
projects. Even worse few distributions install the same versions.
This means that it is nearly impossible for the `pdf2htmlEX` code to
'predict' which version of Poppler or FontForge will be installed on a
given user's machine.
## Our solution
While we *could* keep multiple versions of the `pdf2htmlEX` source code,
each version matched to a particular distribution's installed versions of
Poppler and FontForge, this would be a logistic and testing 'nightmare'.
Instead, when building `pdf2htmlEX`, we download specific versions of both
the Poppler and FontForge sources (usually the most recent), and then
compile *static* versions of the Poppler and FontForge libraries which are
then *statically* linked into the `pdf2htmlEX` binary.
This means that the `pdf2htmlEX` binary is completely independent of any
locally installed versions of either Poppler or FontForge.
However, to get the matched versions of Poppler and FontForge and then
compile them statically, *our* build process becomes much more complex
than a "simple", `configure, make, make install` cycle.
Hence this directory has a large number of shell scripts each of which
automate one simple step in the overall our build process.
## The gory details ...
The shell scripts in this directory automate the download, build, install,
test and upload steps required to provide a complete build/test/release
cycle of `pdf2htmlEX`.
Each script can be used individually to re-run a particular step if needed.
### Top-level scripts
Typically, most users, will run one of the following "top-level" scripts:
1. **`buildInstallLocallyApt`** (**`buildInstallLocallyAlpine`**)
This will automate:
1. the installation of all required development tools
and libraries,
2. download and statically compile the required versions of both
Poppler and FontForge,
3. compile and install `pdf2htmlEX`.
The `*Apt` script will build on any machine which uses the
`apt`/`apt-get` command.
The `*Alpine` script will build on any machine which uses the
`apk` command (Alpine).
2. **`createImagesApt`** (**`createImagesAlpine`**)
Following a successful `buildInstallLocallyApt`, the `createImagesApt`
shell script will create the following images:
1. AppImage
2. Docker image
3. Debian archive
Following a successful `buildInstallLocallyAlpine`, the
`createImagesAlpine` shell script will create the following images:
1. Alpine tar file
2. Docker image
3. **`runTests`**
Following a successful `buildInstallLocallyApt` (or
`buildInstallLocallyAlpine` ), the `runTests` shell script will run the
various 'local' tests reporting errors as they occur.
When run in [Travis-ci](https://travis-ci.org/), failing browser tests
will *not* fail the overall Travis build, but will instead upload the
test results to the GitHub Release page for later review.
4. **`uploadImages`**
Following successful `buildInstallLocally`, `createImages` and
`runTests`, this will automate the upload of the various artefacts to
the `pdf2htmlEX` releases page, and docker hub repository.
**Note** that this step requires the user to enter passwords for each
of the respective services. *Most* users will not need (or be able) to
run this step.
5. **`travisLinuxDoItAll`**
This script is used by the `.travis.yml` configuration to build, test
and upload a complete `pdf2htmlEX` release cycle. It is essentially a
compendium of all of the build scripts in the correct order.
### Individual steps
- **`buildFontforge`**: Compiles a *static* version of `libfontforge` for
use by `pdf2htmlEX`.
Statically linking `libfontforge` into `phd2htmlEX` ensures that any
versions of FontForge already installed by the user, are not broken by
the user's installation of `pdf2htmlEX`.
- **`buildPdf2htmlEX`**: Compiles and links `pdf2htmlEX`.
- **`buildPoppler`**: Compiles a *static* version of `libpoppler` and
`libpopper-glib` for use by `pdf2htmlEX`.
Statically linking `libpoppler` and `libpoppler-glib` into `phd2htmlEX`
ensures that any versions of Poppler already installed by the user, are
not broken by the user's installation of `pdf2htmlEX`.
- **`createAlpineTarFile`**: Using an already compiled version of `pdf2htmlEX`,
installs it and `popper-data` into a tar file suitable for use in any
Alpine environment.
- **`createAppImage`**: Using an already compiled version of `pdf2htmlEX`,
installs it and `popper-data` into an AppImage.
- **`createDebianPackage`**: Using an already compiled version of
`pdf2htmlEX`, installs it and `poppler-data` into a Debian archive
(`*.deb`).
- **`createDockerAlpineImageFromTarFile`**: Installs the Alpine tar file
archive of `pdf2htmlEX` created by `createAlpineTarFile` into an Alpine
Docker image.
- **`createDockerUbuntuImageFromDeb`**: Installs the Debian archive of
`pdf2htmlEX` created by `createDebianPackage` into a Docker image.
- **`getBuildToolsAlpine`**: Locally `apk` installs all development
*tools* required to build `pdf2htmlEX`.
- **`getBuildToolsApt`**: Locally `apt` installs all development *tools*
required to build `pdf2htmlEX`.
- **`getDevLibrariesAlpine`**: Locally `apk` installs all development
*libraries* required to build `pdf2htmlEX`.
- **`getDevLibrariesApt`**: Locally `apt` installs all development
*libraries* required to build `pdf2htmlEX`.
This script provides a definitive list of all libraries required to run
`pdf2htmlEX`.
This script provides a definitive list of all libraries required to run
`pdf2htmlEX`.
- **`getFontforge`**: Downloads and unpacks the version of FontForge specified in the
`FONTFORGE_VERSION` environment variable into the
`pdf2htmlEX/fontoforge` directory.
The `FONTFORGE_VERSION` variable is specified in the `versionEnvs`
script.
- **`getPoppler`**: Downloads and unpacks the version of Poppler specified in the
`POPPLER_VERSION` environment variable into the `pdf2htmlEX/poppler`
directory.
The `POPPLER_VERSION` variable is specified in the `versionEnvs` script.
The `getPoppler` script also downloads and unpacks the most recent
version of `poppler-data`. Since `poppler-data` does not change very
often, the correct version of `poppler-data` is specified in the
`getPoppler` script itself.
- **`installPdf2htmlEX`**: Installs an already compiled version of
`pdf2htmlEX` and `poppler-data` into the location specified by the
`PDF2HTMLEX_PREFIX` environment variable.
The `PDF2HTMLEX_PREFIX` variable is specified in the `versionEnvs`
script.
- **`runTests`**: Runs the tests located in the
`pdf2htmlEX/pdf2htmlEX/test` directory. See the
`pdf2htmlEX/pdf2htmlEx/test` directory's Readme file for details.
- **`uploadDockerImage`**: Upload the `pdf2htmlEX` Docker image to
Docker hub repository associated to the docker hub users specified in
the `DOCKER_USERNAME` environement variable.
Unless the `DOCKER_USERNAME` and `DOCKER_PASSWORD` environment variables
are pre-defined, this script will prompt the user for the respective
values.
- **`uploadGitHubRelease`**: Upload the `pdf2htmlEX` artefacts (AppImage,
Debian archive, test results, etc) to the *continuous* section of the
release page associated with the `TRAVIS_REPO_SLUG` (user/project)
environment variable.
Unless the `GITHUB_USERNAME`, `GITHUB_TOKEN`, and `TRAVIS_REPO_SLUG`
(user/project) environment variables are pre-defined, this script will
prompt the user for the respective values.
### Helper files and scripts
- **`versionEnvs`**: Specifies all of the evnironment variables required
for a standard build of `pdf2htmlEX`. Changes in this script effect
*all* of the other build scripts.
- **`reSourceVersionEnvs`**: This shell script is automatically generated
by the build scripts as they are run. It records the values of all
important environment variables required by the buildScripts. It is
typcically `source`d by each script before it preforms any actions.
- **`reportEnvs`**: Echos all important enviroment variables to the
console. This script is used by the top-level scripts to ensure the
current environment variables are listed before each build.
- **`uploadGitHubReleaseDSL`**: A collection of shell functions used by the
`uploadGitHubRelease` script to automate the upload of release artefacts.
- **`uploadGitHubReleaseMessage`**: The contents of this *text* file is
used by the `uploadGitHubRelease` script as the contents of the release
message, as visible to the user, for the 'continuous' release section.
- **`listFilesByChangeTime`**: A simple shell script which lists the files
in the buildScripts directory by most recently changed files first.
- **`Readme.md`**: This read me file.
## Yet more details?
The various shell script files are meant to be fairly readable. They
contain additional comments about what each step is meant to be doing.

7
buildScripts/buildFontforge
View File

@ -1,8 +1,9 @@
#!/bin/bash
#!/bin/sh
# This bash script build FontForge
# This shell script build FontForge
source buildScripts/reSourceVersionEnvs
# source buildScripts/reSourceVersionEnvs
. buildScripts/reSourceVersionEnvs
echo ""
echo "-------------------------------------------------------------------"

16
buildScripts/buildInstallLocally → buildScripts/buildInstallLocallyAlpine
View File

@ -1,6 +1,6 @@
#!/bin/bash
#!/bin/sh
# This bash script builds the complete pdf2htmlEX application LOCALLY
# This shell script builds the complete pdf2htmlEX application LOCALLY
# (It does not create the AppImage or Docker images)
# Adjust the following two environment variables to suit your needs
@ -32,9 +32,9 @@ set -ev
./buildScripts/reportEnvs
./buildScripts/getBuildTools
./buildScripts/getBuildToolsAlpine
./buildScripts/getDevLibraries
./buildScripts/getDevLibrariesAlpine
./buildScripts/getPoppler
@ -47,11 +47,3 @@ set -ev
./buildScripts/buildPdf2htmlEX
./buildScripts/installPdf2htmlEX
#./buildScripts/createAppImage
#./buildScripts/createDockerImage
#./buildScripts/uploadGitHubRelease
#./buildScripts/uploadDockerImage

49
buildScripts/buildInstallLocallyApt Executable file
View File

@ -0,0 +1,49 @@
#!/bin/sh
# This shell script builds the complete pdf2htmlEX application LOCALLY
# (It does not create the AppImage or Docker images)
# Adjust the following two environment variables to suit your needs
#
export UNATTENDED="--assume-yes"
export MAKE_PARALLEL="-j $(nproc)"
# choose one of the following...
#
# export PDF2HTMLEX_BRANCH="<<YourTagHereWithNoSpaces>>"
export PDF2HTMLEX_BRANCH="$(git rev-parse --abbrev-ref HEAD)"
# The following environment variable determines where the poppler,
# poppler-data, fontforge and pdf2htmlEX packages are installed.
# CHANGE IT TO SUIT YOUR NEEDS:
#
export PDF2HTMLEX_PREFIX=/usr/local
# Ensure all Apt packages are installed with no user interaction
#
export DEBIAN_FRONTEND=noninteractive
set -ev
################
# do the build
./buildScripts/versionEnvs
./buildScripts/reportEnvs
./buildScripts/getBuildToolsApt
./buildScripts/getDevLibrariesApt
./buildScripts/getPoppler
./buildScripts/buildPoppler
./buildScripts/getFontforge
./buildScripts/buildFontforge
./buildScripts/buildPdf2htmlEX
./buildScripts/installPdf2htmlEX

5
buildScripts/buildPdf2htmlEX
View File

@ -1,6 +1,6 @@
#!/bin/bash
#!/bin/sh
# This bash script builds pdf2htmlEX
# This shell script builds pdf2htmlEX
echo ""
echo "-------------------------------------------------------------------"
@ -12,6 +12,7 @@ echo ""
set -ev
cd pdf2htmlEX
rm -rf build
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=$PDF2HTMLEX_PREFIX ..

4
buildScripts/buildPoppler
View File

@ -1,6 +1,6 @@
#!/bin/bash
#!/bin/sh
# This bash script builds the latest poppler
# This shell script builds the latest poppler
echo ""
echo "-------------------------------------------------------------------"

6
buildScripts/cleanPdf2htmlEX
View File

@ -1,6 +0,0 @@
#!/bin/bash
# This bash script clean all pdf2htmlEX build products
cd pdf2htmlEX
rm -rf build pdf2htmlEX.1 share/*.css share/*.js share/*.min.* src/pdf2htmlEX-config.h src/util/css_const.h

80
buildScripts/createAlpineTarFile Executable file
View File

@ -0,0 +1,80 @@
#!/bin/sh
# This shell script creates a docker image from an existing pdf2htmlEX
#
# This is the part which can run *inside* a docker container.
echo ""
echo "-------------------------------------------------------------------"
echo "CREATING pdf2htmlEX Docker Image (run inside a docker container)"
echo " (based on Alpine linux $BUILD_DIST)"
echo "-------------------------------------------------------------------"
echo ""
# Collect everything that will be needed...
# source buildScripts/reSourceVersionEnvs
. buildScripts/reSourceVersionEnvs
export ALPINE_NAME="pdf2htmlEX-$PDF2HTMLEX_NAME"
echo "export ALPINE_NAME=\"$ALPINE_NAME\"" >> buildScripts/reSourceVersionEnvs
set -ev
mkdir -p imageBuild/alpineTarDir
cd pdf2htmlEX/build
sudo rm -rf install_manifest.txt
make install DESTDIR=../../imageBuild/alpineTarDir
cd ../../poppler-data
make install \
prefix=$PDF2HTMLEX_PREFIX \
datadir=$PDF2HTMLEX_PREFIX/share/pdf2htmlEX \
DESTDIR=../imageBuild/alpineTarDir
cd ../imageBuild
tar czvf $ALPINE_NAME.tar.gz alpineTarDir
cat <<ALPINE_INSTALL > $ALPINE_NAME.install
#!/bin/sh
# This (alpine) shell script installs a locally existing tar.gz file of
# the pdf2htmlEX binaries compiled on Alpine $DIST.
#
# You MUST have root/sudo privileges to run this file.
#
# We start by installing all of the (known) required runtime dependencies
#
apk update
#
apk add --no-cache \
tar \
libstdc++ \
libgcc \
gnu-libiconv \
gettext \
glib \
freetype \
fontconfig \
cairo \
libpng \
libjpeg-turbo \
libxml2
# Now we install the (Alpine $DIST) compiled pdf2htmlEX binaries and
# configuration files.
#
cd /
#
tar xvf $ALPINE_NAME.tar.gz --strip-comonents=1
ALPINE_INSTALL
chmod a+x $ALPINE_NAME.tar.gz

22
buildScripts/createAppImage
View File

@ -1,8 +1,9 @@
#!/bin/bash
#!/bin/sh
# This bash script creates an AppImage for pdf2htmlEX
# This shell script creates an AppImage for pdf2htmlEX
source ./buildScripts/reSourceVersionEnvs
# source ./buildScripts/reSourceVersionEnvs
. ./buildScripts/reSourceVersionEnvs
echo ""
echo "-------------------------------------------------------------------"
@ -14,8 +15,8 @@ set -ev
# For appimage output plugin
#
export VERSION="$PDF2HTMLEX_BRANCH-$BUILD_TIME"
export APPIMAGE_NAME="pdf2htmlEX-$PDF2HTMLEX_BRANCH-$BUILD_TIME-$MACHINE_ARCH.AppImage"
#export VERSION="$PDF2HTMLEX_BRANCH-$BUILD_TIME"
export APPIMAGE_NAME="pdf2htmlEX-$PDF2HTMLEX_NAME.AppImage"
echo "export APPIMAGE_NAME=\"$APPIMAGE_NAME\"" >> buildScripts/reSourceVersionEnvs
@ -36,6 +37,14 @@ make install \
cd ../imageBuild
# Make sure directories can be traversed by nobody
#
#find appDir -type d -exec chmod 755 {} \;
#
# Make sure files can be read by nobody
#
#find appDir -type f -exec chmod 644 {} \;
# force libfontconfig into AppImage (linuxdeploy blacklists libfontconfig)
# (turned off since libfontconfig needs to be matched to the underlying
# OS's collection of fonts and /etc/fonts configuration files)
@ -59,6 +68,9 @@ if [ -f /.dockerenv ] ; then
LINUX_DEPLOY_APP_IMAGE=squashfs-root/AppRun
fi
export VERBOSE=1
export OUTPUT=$APPIMAGE_NAME
./$LINUX_DEPLOY_APP_IMAGE \
-e appDir/$PDF2HTMLEX_PREFIX/bin/pdf2htmlEX \
--create-desktop-file \

27
buildScripts/createDebianPackage
View File

@ -1,8 +1,9 @@
#!/bin/bash
#!/bin/sh
# This bash script creates a (binary) Debian Package Archive for pdf2htmlEX
# This shell script creates a (binary) Debian Package Archive for pdf2htmlEX
source ./buildScripts/reSourceVersionEnvs
# source ./buildScripts/reSourceVersionEnvs
. ./buildScripts/reSourceVersionEnvs
echo ""
echo "-------------------------------------------------------------------"
@ -10,11 +11,9 @@ echo "CREATING pdf2htmlEX (binary) Debian package"
echo "-------------------------------------------------------------------"
echo ""
source /etc/lsb-release
set -ev
export DPKG_NAME="pdf2htmlEX-$PDF2HTMLEX_BRANCH-$BUILD_TIME-$MACHINE_ARCH-$DISTRIB_CODENAME.deb"
export DPKG_NAME="pdf2htmlEX-$PDF2HTMLEX_NAME.deb"
echo "export DPKG_NAME=\"$DPKG_NAME\"" >> buildScripts/reSourceVersionEnvs
@ -69,7 +68,7 @@ find $DEBDIR -type f | xargs md5sum > $md5sumsFile
# Accumulate the control file information
#
versionValue=$(git describe --abbrev=0)
versionValue=$PDF2HTMLEX_VERSION.$PDF2HTMLEX_BRANCH.$BUILD_DIST.$BUILD_DATE
architectureValue=$(dpkg-architecture -q DEB_BUILD_ARCH_CPU)
maintainerValue="$(git config --get user.name) <$(git config --get user.email)>"
@ -77,7 +76,7 @@ maintainerValue="$(git config --get user.name) <$(git config --get user.email)>"
#
echo "Package: pdf2htmlEX" > $controlFile
echo "Version: 0:0.$versionValue-0" >> $controlFile
echo "Distribution: $DISTRIB_CODENAME" >> $controlFile
echo "Distribution: $BUILD_DIST" >> $controlFile
echo "Architecture: $architectureValue" >> $controlFile
echo "Section: universe/web" >> $controlFile
echo "Priority: optional" >> $controlFile
@ -96,6 +95,18 @@ touch $conffilesFile
#
cd imageBuild
#
# Make sure directories can be traversed by nobody
#
#find debianDir -type d -exec chmod 755 {} \;
#
# Make sure files can be read by nobody
#
#find debianDir -type f -exec chmod 644 {} \;
#
# Make sure root:root owns all files
#
sudo chown -R root:root debianDir
#
# Build the package
#
dpkg --build debianDir $DPKG_NAME

66
buildScripts/createDockerAlpineImageFromTarFile Executable file
View File

@ -0,0 +1,66 @@
#!/bin/sh
# This shell script creates a docker image from an existing pdf2htmlEX
#
# This is the part which must be run *outside* of any docker container.
echo ""
echo "-------------------------------------------------------------------"
echo "CREATING pdf2htmlEX Docker Image (run outside any docker container)"
echo " (based on Alpine linux $BUILD_DIST)"
echo "-------------------------------------------------------------------"
echo ""
# Collect everything that will be needed...
# source buildScripts/reSourceVersionEnvs
. buildScripts/reSourceVersionEnvs
set -ev
mkdir -p imageBuild/dockerDir
cd imageBuild/dockerDir
cp ../$ALPINE_NAME.* .
if [ -z "$DOCKER_FROM" ]; then
echo ""
read -p "Enter the docker image for the 'from' base: " DOCKER_FROM
echo ""
if [ -z "$DOCKER_FROM" ]; then
echo "DOCKER_FROM not set... so we can not build the docker image"
exit 1
fi
fi
if [ -z "$DOCKER_USERNAME" ]; then
echo ""
read -p "Enter a docker username: " DOCKER_USERNAME
echo ""
if [ -z "$DOCKER_USERNAME" ]; then
echo "DOCKER_USERNAME not set... so we can not build the docker image"
exit 1
fi
fi
export DOCKER_NAME="$DOCKER_USERNAME/pdf2htmlex:$PDF2HTMLEX_NAME"
echo "export DOCKER_FROM=\"$DOCKER_FROM\"" >> ../../buildScripts/reSourceVersionEnvs
echo "export DOCKER_USERNAME=\"$DOCKER_USERNAME\"" >> ../../buildScripts/reSourceVersionEnvs
echo "export DOCKER_NAME=\"$DOCKER_NAME\"" >> ../../buildScripts/reSourceVersionEnvs
cat > Dockerfile <<DOCKERFILE_HERE_DOC
FROM $DOCKER_FROM
COPY ./$ALPINE_NAME.* /root && \
chmod a+x /root/$APLINE_NAME.install
RUN /root/$ALPINE_NAME.install
ENTRYPOINT ["$PDF2HTMLEX_PREFIX/bin/pdf2htmlEX"]
DOCKERFILE_HERE_DOC
cd ..
docker build -t $DOCKER_NAME dockerDir

84
buildScripts/createDockerImage
View File

@ -1,84 +0,0 @@
#!/bin/bash
# This bash script creates a docker image from an existing pdf2htmlEX
echo ""
echo "-------------------------------------------------------------------"
echo "CREATING pdf2htmlEX Docker Image"
echo "-------------------------------------------------------------------"
echo ""
# Collect everything that will be needed...
source buildScripts/reSourceVersionEnvs
source buildScripts/dockerFunctions
set -ev
mkdir -p imageBuild/dockerDir
cd pdf2htmlEX/build
sudo rm -rf install_manifest.txt
make install DESTDIR=../../imageBuild/dockerDir
cd ../../poppler-data
make install \
prefix=$PDF2HTMLEX_PREFIX \
datadir=$PDF2HTMLEX_PREFIX/share/pdf2htmlEX \
DESTDIR=../imageBuild/dockerDir
cd ../imageBuild/dockerDir
copy_deps
# Now. IF we have docker, THEN build the docker image...
if [ -x "$(which docker)" ]; then
if [ -z "$DOCKER_FROM" ]; then
echo ""
read -p "Enter the docker image for the 'from' base: " DOCKER_FROM
echo ""
if [ -z "$DOCKER_FROM" ]; then
echo "DOCKER_FROM not set... so we can not build the docker image"
exit 1
fi
fi
if [ -z "$DOCKER_USERNAME" ]; then
echo ""
read -p "Enter a docker username: " DOCKER_USERNAME
echo ""
if [ -z "$DOCKER_USERNAME" ]; then
echo "DOCKER_USERNAME not set... so we can not build the docker image"
exit 1
fi
fi
export DOCKER_NAME="$DOCKER_USERNAME/pdf2htmlex:$PDF2HTMLEX_BRANCH-$BUILD_TIME"
echo "export DOCKER_FROM=\"$DOCKER_FROM\"" >> buildScripts/reSourceVersionEnvs
echo "export DOCKER_USERNAME=\"$DOCKER_USERNAME\"" >> buildScripts/reSourceVersionEnvs
echo "export DOCKER_NAME=\"$DOCKER_NAME\"" >> buildScripts/reSourceVersionEnvs
delete_blacklisted
cat > Dockerfile <<DOCKERFILE_HERE_DOC
FROM $DOCKER_FROM
COPY ./ /
RUN ldconfig
ENTRYPOINT ["$PDF2HTMLEX_PREFIX/bin/pdf2htmlEX"]
DOCKERFILE_HERE_DOC
cd ..
docker build -t $DOCKER_NAME dockerDir
else
echo "Docker is not installed... skipping creation of a docker image."
fi

69
buildScripts/createDockerUbuntuImageFromDeb Executable file
View File

@ -0,0 +1,69 @@
#!/bin/sh
# This shell script creates a docker image from an existing pdf2htmlEX
#
# This is the part which must be run *outside* of any docker container.
echo ""
echo "-------------------------------------------------------------------"
echo "CREATING pdf2htmlEX Docker Image (from deb archive)"
echo "-------------------------------------------------------------------"
echo ""
# Collect everything that will be needed...
# source buildScripts/reSourceVersionEnvs
. buildScripts/reSourceVersionEnvs
set -ev
cd imageBuild
mkdir -p dockerDir
cp $DPKG_NAME dockerDir
cd dockerDir
if [ -z "$DOCKER_FROM" ]; then
echo ""
read -p "Enter the docker image for the 'from' base: " DOCKER_FROM
echo ""
if [ -z "$DOCKER_FROM" ]; then
echo "DOCKER_FROM not set... so we can not build the docker image"
exit 1
fi
fi
if [ -z "$DOCKER_USERNAME" ]; then
echo ""
read -p "Enter a docker username: " DOCKER_USERNAME
echo ""
if [ -z "$DOCKER_USERNAME" ]; then
echo "DOCKER_USERNAME not set... so we can not build the docker image"
exit 1
fi
fi
export DOCKER_NAME="$DOCKER_USERNAME/pdf2htmlex:$PDF2HTMLEX_NAME"
echo "export DOCKER_FROM=\"$DOCKER_FROM\"" >> ../../buildScripts/reSourceVersionEnvs
echo "export DOCKER_USERNAME=\"$DOCKER_USERNAME\"" >> ../../buildScripts/reSourceVersionEnvs
echo "export DOCKER_NAME=\"$DOCKER_NAME\"" >> ../../buildScripts/reSourceVersionEnvs
cat > Dockerfile <<DOCKERFILE_HERE_DOC
FROM $DOCKER_FROM
COPY ./$DPKG_NAME /root
RUN apt update && \
apt -y upgrade && \
apt -y --no-install-recommends install \
/root/$DPKG_NAME
ENTRYPOINT ["$PDF2HTMLEX_PREFIX/bin/pdf2htmlEX"]
DOCKERFILE_HERE_DOC
cd ..
docker build -t $DOCKER_NAME dockerDir

18
buildScripts/createImages
View File

@ -1,18 +0,0 @@
#!/bin/bash
set -ev
# This bash script creates the pdf2htmlEX AppImage and Docker Images
#################
# do the creation
./buildScripts/reportEnvs || { echo 'reportEnvs FAILED' ; exit 1 ; }
./buildScripts/createAppImage || { echo 'createAppImage FAILED' ; exit 1 ; }
./buildScripts/createDockerImage || { echo 'createDockerImage FAILED' ; exit 1 ; }
./buildScripts/createDebianPackage || { echo 'createDebianPackage FAILED' ; exit 1 ; }

19
buildScripts/createImagesAlpine Executable file
View File

@ -0,0 +1,19 @@
#!/bin/sh
set -ev
# This shell script creates the pdf2htmlEX AppImage and Docker Images
#################
# do the creation
./buildScripts/reportEnvs
./buildScripts/createAlpineTarFile
if [ -x "$(which docker)" ]; then
./buildScripts/createDockerAlpineImageFromTarFile
fi

21
buildScripts/createImagesApt Executable file
View File

@ -0,0 +1,21 @@
#!/bin/sh
set -ev
# This shell script creates the pdf2htmlEX AppImage and Docker Images
#################
# do the creation
./buildScripts/reportEnvs
./buildScripts/createAppImage
./buildScripts/createDebianPackage
if [ -x "$(which docker)" ]; then
./buildScripts/createDockerUbuntuImageFromDeb
fi

59
buildScripts/dockerFunctions
View File

@ -1,59 +0,0 @@
# The following has been extracted and modified for our use under an MIT
# license by Stephen Gaito on 2019/NOV/25
#
# The original was taken from:
# https://raw.githubusercontent.com/AppImage/pkg2appimage/master/functions.sh
# (commit ID ed1d385282a6aa6c9a93b52296f20555adf9bae7 commited 2019/Apr/9)
#
# The original copyright was: Copyright (c) 2004-16 Simon Peter
#
# Copy the library dependencies of all exectuable files in the current
# directory (it can be beneficial to run this multiple times)
#
copy_deps()
{
PWD=$(readlink -f .)
FILES=$(find . -type f -executable -or -name *.so.* -or -name *.so \
| sort | uniq )
for FILE in $FILES ; do
ldd "${FILE}" | grep "=>" | awk '{print $3}' \
| xargs -I '{}' echo '{}' >> DEPSFILE
done
DEPS=$(cat DEPSFILE | sort | uniq)
for FILE in $DEPS ; do
if [ -e $FILE ] && [[ $(readlink -f $FILE)/ != $PWD/* ]] ; then
cp -v --parents -rfL $FILE ./ || true
fi
done
rm -f DEPSFILE
}
# Delete blacklisted files. Our blacklisted files are the shared libraries
# extracted from the "base" docker image
#
delete_blacklisted()
{
docker run --rm -it --entrypoint=find $DOCKER_FROM -iname "*.so.*" \
| xargs -i basename {} | sed 's/\r//g' | sed 's/\(so\.[0-9]\+\).*$/\1/' \
| sort | uniq > BLACKLISTED_FILES
for FILE in $(cat BLACKLISTED_FILES | tr "\r\n" " ") ; do
FILES="$(find . -name "${FILE}*" -not -path "./usr/optional/*")"
for FOUND in $FILES ; do
rm -vf "$FOUND" "$(readlink -f "$FOUND")"
done
done
rm BLACKLISTED_FILES
# Do not bundle developer stuff
rm -rf usr/include || true
rm -rf usr/lib/cmake || true
rm -rf usr/lib/pkgconfig || true
find . -name '*.la' | xargs -i rm {}
}

30
buildScripts/getBuildToolsAlpine Executable file
View File

@ -0,0 +1,30 @@
#!/bin/sh
# This shell script automates getting the required build tools (apt install)
# set the shell environment variable 'UNATTENDED' to '--assume-yes' for
# unattended use (for example in the .travis.yml script)
echo ""
echo "-------------------------------------------------------------------"
echo "INSTALLING Build Tools (using APK / Alpine)"
echo "-------------------------------------------------------------------"
echo ""
set -ev
sudo apk update
sudo apk add \
sudo \
wget \
git \
pkgconfig \
ruby \
cmake \
make \
gcc \
g++ \
gettext \
openjdk8 \
jq

12
buildScripts/getBuildTools → buildScripts/getBuildToolsApt
View File

@ -1,8 +1,8 @@
#!/bin/bash
#!/bin/sh
# This bash script automates getting the required build tools (apt install)
# This shell script automates getting the required build tools (apt install)
# set the bash environment variable 'UNATTENDED' to '--assume-yes' for
# set the shell environment variable 'UNATTENDED' to '--assume-yes' for
# unattended use (for example in the .travis.yml script)
echo ""
@ -16,7 +16,8 @@ set -ev
sudo apt-get update
sudo apt-get $UNATTENDED install \
aptitude \
sudo \
wget \
git \
pkg-config \
ruby \
@ -30,6 +31,5 @@ sudo apt-get $UNATTENDED install \
dpkg-dev \
gettext \
openjdk-8-jre-headless \
jq \
tree
jq

28
buildScripts/getDevLibrariesAlpine Executable file
View File

@ -0,0 +1,28 @@
#!/bin/sh
# This shell script automates getting the development libraries required to
# build poppler and fontforge
# set the shell environment variable 'UNATTENDED' to '--assume-yes' for
# unattended use (for example in the .travis.yml script)
echo ""
echo "-------------------------------------------------------------------"
echo "INSTALLING development libraries (using APK on Alpine)"
echo "-------------------------------------------------------------------"
echo ""
set -ev
sudo apk update
sudo apk add \
gettext \
gnu-libiconv-dev \
cairo-dev \
libpng-dev \
freetype-dev \
gettext-dev \
glib-dev \
fontconfig-dev \
libjpeg-turbo-dev \
libxml2-dev

6
buildScripts/getDevLibraries → buildScripts/getDevLibrariesApt
View File

@ -1,9 +1,9 @@
#!/bin/bash
#!/bin/sh
# This bash script automates getting the development libraries required to
# This shell script automates getting the development libraries required to
# build poppler and fontforge
# set the bash environment variable 'UNATTENDED' to '--assume-yes' for
# set the shell environment variable 'UNATTENDED' to '--assume-yes' for
# unattended use (for example in the .travis.yml script)
echo ""

7
buildScripts/getFontforge
View File

@ -1,8 +1,9 @@
#!/bin/bash
#!/bin/sh
# This bash script gets and unpacks the latest fontforge AppImage
# This shell script gets and unpacks the latest fontforge AppImage
source buildScripts/reSourceVersionEnvs
# source buildScripts/reSourceVersionEnvs
. buildScripts/reSourceVersionEnvs
echo ""
echo "-------------------------------------------------------------------"

25
buildScripts/getPdf2htmlEX
View File

@ -1,25 +0,0 @@
#!/bin/bash
# This bash script automates the process of getting the original pdf2htmlEX
# source
PDF2HTMLEX_BRANCH=update-poppler
echo ""
echo "-------------------------------------------------------------------"
echo "GETTING pdf2htmlEX sources (using wget)"
echo " (PDF2HTMLEX_BRANCH: [$PDF2HTMLEX_BRANCH])"
echo "-------------------------------------------------------------------"
echo ""
set -ev
wget https://codeload.github.com/stephengaito/pdf2htmlEX/zip/$PDF2HTMLEX_BRANCH
mv $PDF2HTMLEX_BRANCH $PDF2HTMLEX_BRANCH.zip
sudo apt install unzip
unzip $PDF2HTMLEX_BRANCH.zip
mv pdf2htmlEX-$PDF2HTMLEX_BRANCH pdf2htmlEX

7
buildScripts/getPoppler
View File

@ -1,8 +1,9 @@
#!/bin/bash
#!/bin/sh
# This bash script gets and unpacks the latest Poppler source code
# This shell script gets and unpacks the latest Poppler source code
source buildScripts/reSourceVersionEnvs
# source buildScripts/reSourceVersionEnvs
. buildScripts/reSourceVersionEnvs
echo ""
echo "-------------------------------------------------------------------"

2
buildScripts/installPdf2htmlEX
View File

@ -1,4 +1,4 @@
#!/bin/bash
#!/bin/sh
echo ""
echo "-------------------------------------------------------------------"

2
buildScripts/listFilesByChangeTime
View File

@ -1,4 +1,4 @@
#!/bin/bash
#!/bin/sh
# see:https://stackoverflow.com/a/7448828

13
buildScripts/reportEnvs
View File

@ -1,8 +1,12 @@
#!/bin/bash
#!/bin/sh
# This bash script reports the more important TRAVIS environment variables
# This shell script reports the more important TRAVIS environment variables
# source ./buildScripts/reSourceVersionEnvs
. ./buildScripts/reSourceVersionEnvs
echo "TravisCI env:"
echo " dist: [$TRAVIS_DIST]"
echo " tag: [$TRAVIS_TAG]"
echo " branch: [$TRAVIS_BRANCH]"
echo " commit: [$TRAVIS_COMMIT]"
@ -13,17 +17,20 @@ echo "Build env:"
echo " prefix: [$PDF2HTMLEX_PREFIX]"
echo " unattended: [$UNATTENDED]"
echo " make parallel: [$MAKE_PARALLEL]"
source ./buildScripts/reSourceVersionEnvs
echo " distribution: [$BUILD_DIST]"
echo "pdf2htmlEX env:"
echo " version: [$PDF2HTMLEX_VERSION]"
echo " poppler: [$POPPLER_VERSION]"
echo " fontforge: [$FONTFORGE_VERSION]"
echo " pdf2htmlEX: [$PDF2HTMLEX_BRANCH]"
echo " machine arch: [$MACHINE_ARCH]"
echo "release env:"
echo " build date: [$BUILD_DATE]"
echo " build time: [$BUILD_TIME]"
echo " docker from: [$DOCKER_FROM]"
echo " docker name: [$DOCKER_NAME]"
echo " appImage name: [$APPIMAGE_NAME]"
echo " dpkg name: [$DPKG_NAME]"
echo "uploadTool suffix: [$UPLOADTOOL_SUFFIX]"

33
buildScripts/runTests Executable file
View File

@ -0,0 +1,33 @@
#!/bin/sh
# source ./buildScripts/reSourceVersionEnvs
. ./buildScripts/reSourceVersionEnvs
echo ""
echo "-------------------------------------------------------------------"
echo "RUNNING pdf2htmlEX tests"
echo "-------------------------------------------------------------------"
echo ""
export TEST_RESULTS_NAME="pdf2htmlEX-$PDF2HTMLEX_BRANCH-$BUILD_TIME-$BUILD_DIST-$MACHINE_ARCH-testResults.zip"
echo "export TEST_RESULTS_NAME=\"$TEST_RESULTS_NAME\"" >> buildScripts/reSourceVersionEnvs
set -ev
# This shell script runs the pdf2htmlEX tests
cd pdf2htmlEX/test
# The following is only needed for the local browser tests
#
./installAutomaticTestSoftwareApt
./runLocalTestsShell
./runLocalBrowserTests
cd ../..
mkdir -p imageBuild
zip -r imageBuild/$TEST_RESULTS_NAME /tmp/pdf2htmlEX/html /tmp/pdf2htmlEX/png

17
buildScripts/travisLinuxDoItAll
View File

@ -1,16 +1,19 @@
#!/bin/bash
#!/bin/sh
# This bash script builds everyting on an TravisCI Linux (Ubunutu) worker
# This shell script builds everyting on an TravisCI Linux (Ubunutu) worker
set -ev
export UNATTENDED="--assume-yes"
export MAKE_PARALLEL="-j $(nproc)"
export PDF2HTMLEX_BRANCH="$(git rev-parse --abbrev-ref HEAD)"
export PDF2HTMLEX_BRANCH=$TRAVIS_BRANCH
export PDF2HTMLEX_PREFIX=/usr/local
export PDF2HTMLEX_PATH=/usr/local/bin/pdf2htmlEX
# Ensure all Apt packages are installed with no user interaction
#
export DEBIAN_FRONTEND=noninteractive
@ -20,17 +23,17 @@ export DEBIAN_FRONTEND=noninteractive
./buildScripts/versionEnvs
./buildScripts/reportEnvs
./buildScripts/getBuildTools
./buildScripts/getDevLibraries
./buildScripts/getBuildToolsApt
./buildScripts/getDevLibrariesApt
./buildScripts/getPoppler
./buildScripts/buildPoppler
./buildScripts/getFontforge
./buildScripts/buildFontforge
./buildScripts/buildPdf2htmlEX
./buildScripts/installPdf2htmlEX
#./testScripts/runLocalTests
./buildScripts/runTests
./buildScripts/createAppImage
./buildScripts/createDebianPackage
./buildScripts/createDockerImage
./buildScripts/createDockerUbuntuImageFromDeb
#./buildScripts/uploadGitHubRelease
#./buildScripts/uploadDockerImage

7
buildScripts/uploadDockerImage
View File

@ -1,6 +1,6 @@
#!/bin/bash
#!/bin/sh
# This bash script uploads the pdf2htmlEX docker image to docker hub
# This shell script uploads the pdf2htmlEX docker image to docker hub
#
# We EXPECT the following environment variables to be set:
# DOCKER_USERNAME
@ -9,7 +9,8 @@
# DOCKER_PASSWORD (if not set you will be asked for your password)
#
source ./buildScripts/reSourceVersionEnvs
# source ./buildScripts/reSourceVersionEnvs
. ./buildScripts/reSourceVersionEnvs
echo ""
echo "-------------------------------------------------------------------"

22
buildScripts/uploadGitHubRelease
View File

@ -1,6 +1,6 @@
#!/bin/bash
#!/bin/sh
# This bash script uploads the pdf2htmlEX release artefacts
# This shell script uploads the pdf2htmlEX release artefacts
#
# We EXPECT the following environment variables to be set:
# GITHUB_TOKEN
@ -10,9 +10,11 @@
# DOCKER_PASSWORD (if not set you will be asked for your password)
#
source ./buildScripts/reSourceVersionEnvs
# source ./buildScripts/reSourceVersionEnvs
. ./buildScripts/reSourceVersionEnvs
cp ./buildScripts/reSourceVersionEnvs imageBuild/buildInfo.sh
source ./buildScripts/uploadGitHubReleaseDSL
. ./buildScripts/uploadGitHubReleaseDSL
# source ./buildScripts/uploadGitHubReleaseDSL
echo ""
echo "-------------------------------------------------------------------"
@ -60,6 +62,8 @@ fi
cd imageBuild
ls -la
echo "machine api.github.com" > $HOME/.netrc
echo " login $GITHUB_USERNAME" >> $HOME/.netrc
echo " password $GITHUB_TOKEN" >> $HOME/.netrc
@ -69,14 +73,20 @@ echo " password $GITHUB_TOKEN" >> $HOME/.netrc
echo $APPIMAGE_NAME > appImageName.txt
echo $DOCKER_NAME > dockerImageName.txt
echo $DPKG_NAME > debianArchiveName.txt
echo $TEST_RESULTS_NAME > testResultsName.txt
deleteReleaseByTag "$TRAVIS_REPO_SLUG" "continuous"
deleteReleaseByTag "$TRAVIS_REPO_SLUG" "continuous-$BUILD_DIST"
createNewRelease "$TRAVIS_REPO_SLUG" "continuous" "Latest release" \
createNewRelease "$TRAVIS_REPO_SLUG" "continuous-$BUILD_DIST" "Latest $BUILD_DIST release" \
../buildScripts/uploadGitHubReleaseMessage
uploadAnAsset $upload_url "appImageName.txt" "text/plain"
uploadAnAsset $upload_url "testResultsName.txt" "text/plain"
uploadAnAsset $upload_url "buildInfo.sh" "text/plain"
uploadAnAsset $upload_url "dockerImageName.txt" "text/plain"
uploadAnAsset $upload_url "debianArchiveName.txt" "text/plain"
uploadAnAsset $upload_url $APPIMAGE_NAME "application/zip"
uploadAnAsset $upload_url $TEST_RESULTS_NAME "application/zip"
uploadAnAsset $upload_url $DPKG_NAME "application/x-debian-package"

10
buildScripts/uploadGitHubReleaseDSL
View File

@ -1,8 +1,8 @@
#!/usr/bin/env bash
#!/usr/bin/env sh
# This bash script is a collection of functions to interact with the GitHub
# release and git/tag APIs. It essentially provides a bash based DSL for
# uploading GitHub release artefacts.
# This shell script is a collection of functions to interact with the GitHub
# release and git/tag APIs. It essentially provides a shell script based DSL
# for uploading GitHub release artefacts.
# NOTE-1: These functions make use of your ".netrc" file to authenticate
# with GitHub. In order to use these functions, make sure you have **both**
@ -171,7 +171,7 @@ function uploadAnAsset {
echo ""
}
# The above has been heavily modified into a collection of bash functions
# The above has been heavily modified into a collection of shell functions
# by Stephen Gaito working on the pdf2htmlEX project.
#
# It has been based upon the code in:

35
buildScripts/uploadGitHubReleaseMessage
View File

@ -1,6 +1,35 @@
This is the development/testing release of the pdf2htmlEX AppImage.
This is the most recent release of the pdf2htmlEX project.
You can download the AppImage and 'just run it'.
We release three types of binary objects:
See: https://appimage.org/ for details
1. The file *.AppImage on this release page is an AppImage
You can download the AppImage, make it executable and then 'just run
it'.
See: https://appimage.org/ for details
2. The file *.deb is a Debian Archive
You can download the Debian Archive and install it locally using:
sudo apt install <<releativePathToDebFile>>
While the Debian archive is built on a specific Ubuntu release, it can
usually be used on any more recent Ubuntu releases.
3. You can alternatively get pdf2htmlEX Docker images from Docker Hub in
either of the repositories:
pdf2htmlEX/pdf2htmlEX (offical release)
or
stephengaito/pdf2htmlEX (development releases)
We also release a zip archive of the browser test results in the
*-testResults.zip file.
Finally, the various *.txt and *buildInfo.sh files have well known
(unchanging) names which can be used by automatic scripts to identifiy the
current full names of the corresponding binary artifacts.

4
buildScripts/uploadImages
View File

@ -1,6 +1,6 @@
#!/bin/bash
#!/bin/sh
# This bash script uploads already existing pdf2htmlEX AppImage and Docker
# This shell script uploads already existing pdf2htmlEX AppImage and Docker
# Images
################

34
buildScripts/versionEnvs
View File

@ -1,11 +1,13 @@
#!/bin/bash
#!/bin/sh
# This bash script exports environment variables for the latest software
# This shell script exports environment variables for the latest software
# versions
# see: https://poppler.freedesktop.org/releases.html
# current working: 0.83.0
export PDF2HTMLEX_VERSION=0.18.8.alpha
export POPPLER_VERSION=poppler-0.89.0
#export POPPLER_VERSION=poppler-0.88.0
#export POPPLER_VERSION=poppler-0.87.0
@ -40,13 +42,37 @@ if [ -z "$PDF2HTMLEX_BRANCH" ]; then
fi
fi
export MACHINE_ARCH="$(uname -m)"
if test -r /etc/lsb-release ; then
cat /etc/lsb-release
# source /etc/lsb-release
. /etc/lsb-release
export BUILD_DIST=$DISTRIB_CODENAME
export BUILD_OS=$DISTRIB_ID
elif test -r /etc/alpine-release ; then
cat /etc/alpine-release
export BUILD_OS=alpine
export BUILD_DIST=$(cat /etc/alpine-release)
else
echo "FAILURE: could not determine release"
exit -1
fi
export BUILD_DATE="$(date +%Y%m%d)"
export BUILD_TIME="$(date +%Y_%m_%d-%H_%M_%S)"
echo "export POPPLER_VERSION=\"$POPPLER_VERSION\"" > buildScripts/reSourceVersionEnvs
export MACHINE_ARCH="$(uname -m)"
export PDF2HTMLEX_NAME=$PDF2HTMLEX_VERSION-$PDF2HTMLEX_BRANCH-$BUILD_DATE-$BUILD_OS-$BUILD_DIST-$MACHINE_ARCH
echo "export PDF2HTMLEX_NAME=\"$PDF2HTMLEX_NAME\"" > buildScripts/reSourceVersionEnvs
echo "export PDF2HTMLEX_VERSION=\"$PDF2HTMLEX_VERSION\"" >> buildScripts/reSourceVersionEnvs
echo "export POPPLER_VERSION=\"$POPPLER_VERSION\"" >> buildScripts/reSourceVersionEnvs
echo "export FONTFORGE_VERSION=\"$FONTFORGE_VERSION\"" >> buildScripts/reSourceVersionEnvs
echo "export PDF2HTMLEX_BRANCH=\"$PDF2HTMLEX_BRANCH\"" >> buildScripts/reSourceVersionEnvs
echo "export MACHINE_ARCH=\"$MACHINE_ARCH\"" >> buildScripts/reSourceVersionEnvs
echo "export BUILD_OS=\"$BUILD_OS\"" >> buildScripts/reSourceVersionEnvs
echo "export BUILD_DIST=\"$BUILD_DIST\"" >> buildScripts/reSourceVersionEnvs
echo "export BUILD_DATE=\"$BUILD_DATE\"" >> buildScripts/reSourceVersionEnvs
echo "export BUILD_TIME=\"$BUILD_TIME\"" >> buildScripts/reSourceVersionEnvs
echo "export PDF2HTMLEX_PREFIX=\"$PDF2HTMLEX_PREFIX\"" >> buildScripts/reSourceVersionEnvs

10
pdf2htmlEX/CMakeLists.txt
View File

@ -79,9 +79,18 @@ set(FONTFORGE_LIBRARIES ${FONTFORGE_LIBRARIES}
${CMAKE_SOURCE_DIR}/../fontforge/build/lib/libfontforge.a
)
# If we are using Alpine Linux then we need to add -lintl
#
if (EXISTS /usr/lib/libintl.so )
set(LIB_INTL_LIBRARIES -lintl )
else ()
set(LIB_INTL_LIBRARIES "" )
endif()
set(PDF2HTMLEX_LIBS ${PDF2HTMLEX_LIBS}
${POPPLER_LIBRARIES}
${FONTFORGE_LIBRARIES}
${LIB_INTL_LIBRARIES}
${CAIRO_LIBRARIES}
-ljpeg
-lpng
@ -262,6 +271,7 @@ set(PDF2HTMLEX_TMPDIR /tmp/pdf2htmlEX/tmp)
set(PDF2HTMLEX_DATDIR /tmp/pdf2htmlEX/dat)
set(PDF2HTMLEX_PNGDIR /tmp/pdf2htmlEX/png)
set(PDF2HTMLEX_OUTDIR /tmp/pdf2htmlEX/out)
set(PDF2HTMLEX_HTMDIR /tmp/pdf2htmlEX/html)
file(MAKE_DIRECTORY ${PDF2HTMLEX_TMPDIR})
file(MAKE_DIRECTORY ${PDF2HTMLEX_DATDIR})
file(MAKE_DIRECTORY ${PDF2HTMLEX_PNGDIR})

4
pdf2htmlEX/pdf2htmlEX.1.in
View File

@ -359,7 +359,7 @@ pdf2htmlEX is written by Lu Wang <coolwanglu@gmail.com>
.SH SEE ALSO
.TP
Home page
https://github.com/coolwanglu/pdf2htmlEX
https://github.com/pdf2htmlEX/pdf2htmlEX
.TP
pdf2htmlEX Wiki
https://github.com/coolwanglu/pdf2htmlEX/wiki
https://github.com/pdf2htmlEX/pdf2htmlEX/wiki

2
pdf2htmlEX/share/base.css.in
View File

@ -2,7 +2,7 @@
/*!
* Base CSS for pdf2htmlEX
* Copyright 2012,2013 Lu Wang <coolwanglu@gmail.com>
* https://github.com/coolwanglu/pdf2htmlEX/blob/master/share/LICENSE
* https://github.com/pdf2htmlEX/pdf2htmlEX/blob/master/share/LICENSE
*/
/* Part 1: Web Page Layout: Free to modify, except for a few of them which are required by pdf2htmlEX.js, see the comments */
#sidebar { /* Sidebar */

2
pdf2htmlEX/share/fancy.css.in
View File

@ -2,7 +2,7 @@
/*!
* Fancy styles for pdf2htmlEX
* Copyright 2012,2013 Lu Wang <coolwanglu@gmail.com>
* https://github.com/coolwanglu/pdf2htmlEX/blob/master/share/LICENSE
* https://github.com/pdf2htmlEX/pdf2htmlEX/blob/master/share/LICENSE
*/
@keyframes fadein { from { opacity:0;} to { opacity:1;} }
@-webkit-keyframes fadein { from { opacity:0;} to { opacity:1;} }

2
pdf2htmlEX/share/manifest
View File

@ -17,7 +17,7 @@
# Declaration - Do not modify
"""
<!DOCTYPE html>
<!-- Created by pdf2htmlEX (https://github.com/coolwanglu/pdf2htmlex) -->
<!-- Created by pdf2htmlEX (https://github.com/pdf2htmlEX/pdf2htmlEX) -->
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta charset="utf-8"/>

2
pdf2htmlEX/share/pdf2htmlEX.js.in
View File

@ -2,7 +2,7 @@
/**
* @license pdf2htmlEX.js: Core UI functions for pdf2htmlEX
* Copyright 2012,2013 Lu Wang <coolwanglu@gmail.com> and other contributors
* https://github.com/coolwanglu/pdf2htmlEX/blob/master/share/LICENSE
* https://github.com/pdf2htmlEX/pdf2htmlEX/blob/master/share/LICENSE
*/
/*

2
pdf2htmlEX/test/README.md
View File

@ -19,7 +19,7 @@ and needs to be fixed. Rerun `runLocalBrowserTests` and use the
## Running tests
There are three bash scripts which automate the running of a given
There are three shell scripts which automate the running of a given
collection of tests:
1. **runLocalTests** runs a simple collection of tests which do not

32
pdf2htmlEX/test/browser_tests.py
View File

@ -37,12 +37,22 @@ class BrowserTests(Common):
ref_htmlfolder = os.path.join(self.TEST_DATA_DIR, basefilename)
ref_htmlfilename = os.path.join(ref_htmlfolder, htmlfilename)
out_htmlfilename = os.path.join(self.OUTDIR, htmlfilename)
pre_htmlfilename = os.path.join(self.PREDIR, htmlfilename)
try:
# see if we have pre-compiled the html file...
# if so simply copy it into place
#
shutil.copy(pre_htmlfilename, out_htmlfilename)
except:
# we have not pre-compiled the html file
# so create it using pdf2htmlEX
#
pdf2htmlEX_args = self.DEFAULT_PDF2HTMLEX_ARGS + args + [
os.path.join(self.TEST_DATA_DIR, filename),
htmlfilename ]
result = self.run_pdf2htmlEX(pdf2htmlEX_args)
#
self.assertIn(htmlfilename, result['output_files'], 'HTML file is not generated')
if self.GENERATING_MODE:
@ -51,6 +61,20 @@ class BrowserTests(Common):
shutil.copytree(self.OUTDIR, ref_htmlfolder)
return
# keep a record of the HTML files (and any differences)
# for later reporting of test results
#
try:
os.makedirs(self.HTMDIR, 0o755, True)
outHtmlFile = os.path.join(self.HTMDIR, basefilename+'.out.html')
refHtmlFile = os.path.join(self.HTMDIR, basefilename+'.ref.html')
difHtmlFile = os.path.join(self.HTMDIR, basefilename+'.diff.html')
shutil.copy(out_htmlfilename, outHtmlFile)
shutil.copy(ref_htmlfilename, refHtmlFile)
os.system("diff "+outHtmlFile+" "+refHtmlFile+" > "+difHtmlFile+" 2>&1")
except:
pass
pngfilename_out = os.path.join(self.PNGDIR, basefilename + '.out.png')
self.generate_image(out_htmlfilename, pngfilename_out)
out_img = Image.open(pngfilename_out).convert('RGB')
@ -70,9 +94,11 @@ class BrowserTests(Common):
print("\nTesting at: [", basefilename, "]")
if diff_bbox is None:
print(" passed")
print(" SUCCESS: ", basefilename)
elif basefilename == "test_fail" :
print(" SUCCESS: ", basefilename)
else:
print(" diff bounding box: ", diff_bbox, " should be None")
print(" FAILURE: ", basefilename, " diff bounding box: ", diff_bbox, " should be None")
diff_size = (diff_bbox[2] - diff_bbox[0]) * (diff_bbox[3] - diff_bbox[1])
img_size = ref_img.size[0] * ref_img.size[1]
self.fail(('PNG files %s and %s differ by at most %d pixels, '+

14
pdf2htmlEX/test/installAutomaticTestSoftware → pdf2htmlEX/test/installAutomaticTestSoftwareApt
View File

@ -1,6 +1,8 @@
#!/bin/bash
#!/bin/sh
# This bash script installs all local software required to run the
set -ev
# This shell script installs all local software required to run the
# pdf2htmlEX tests
export DEBIAN_FRONTEND=noninteractive
@ -8,6 +10,9 @@ export DEBIAN_FRONTEND=noninteractive
# Start by making sure all required apt packages exist
#
sudo apt -y install \
wget \
diffutils \
zip \
python3 \
python3-pip \
xvfb \
@ -15,7 +20,8 @@ sudo apt -y install \
# Now get the geckodriver for firefox (as required by selenium)
#
pushd /tmp
oldPWD=$(pwd)
cd /tmp
#
wget https://github.com/mozilla/geckodriver/releases/download/v0.26.0/geckodriver-v0.26.0-linux64.tar.gz
#
@ -23,7 +29,7 @@ tar xvf geckodriver-v0.26.0-linux64.tar.gz
#
sudo mv geckodriver /usr/local/bin
#
popd
cd $oldPWD
# Now make sure all python packages exist (install into the local user's
# PyPI archive)

4
pdf2htmlEX/test/installManualTestSoftware
View File

@ -1,6 +1,6 @@
#!/bin/bash
#!/bin/sh
# This bash script installs all local software required to run the
# This shell script installs all local software required to run the
# pdf2htmlEX tests
export DEBIAN_FRONTEND=noninteractive

111
pdf2htmlEX/test/produceHtmlForBrowserTests Executable file
View File

@ -0,0 +1,111 @@
#!/bin/sh
# This shell script walks through the browser_tests directory running
# pdf2htmlEX on each *.pdf file.
# This is how we run pdf2htmlEX on a particular file, and arguments.
#
runPdf2htmlEX() {
pdfFileName=$1
htmlFileName=$(echo $pdfFileName | cut -d'.' -f1).html
arguments=$2
#
echo ""
echo "---"
echo " pdfFileName: [$pdfFileName]"
echo "htmlFileName: [$htmlFileName]"
echo " arguments: [$arguments]"
#
# now run pdf2htmlEX to produce the output files
#
echo $PDF2HTMLEX_PATH \
--data-dir=$PDF2HTMLEX_DATDIR \
--dest-dir $PDF2HTMLEX_TMPDIR \
--fit-width=800 --last-page=1 \
$arguments \
browser_tests/$pdfFileName \
$htmlFileName
#
$PDF2HTMLEX_PATH \
--data-dir=$PDF2HTMLEX_DATDIR \
--dest-dir $PDF2HTMLEX_TMPDIR \
--fit-width=800 --last-page=1 \
$arguments \
browser_tests/$pdfFileName \
$htmlFileName
}
# This is how we copy a file omitting lines between '#TEST_IGNORE_BEGIN'
# and '#TEST_IGNORE_END' (we pipe the file in via stdin and save it via
# stdout)
#
copy_TEST_IGNORE_file() {
skipLine=echo
while IFS= read -r line ; do
if echo $line | grep -q "^#TEST_IGNORE_BEGIN" ; then
skipLine=true
elif echo $line | grep -q "^#TEST_IGNORE_END" ; then
skipLine=echo
else
$skipLine "$line"
fi
done
}
if test -z "$PDF2HTMLEX_PATH" ; then
echo "PANIC: we do not know where to find the pdf2htmlEX executable"
exit 1
fi
if test -z "$PDF2HTMLEX_DATDIR" ; then
export PDF2HTMLEX_DATDIR=/tmp/pdf2htmlEX/dat
fi
if test -z "$PDF2HTMLEX_TMPDIR" ; then
export PDF2HTMLEX_TMPDIR=/tmp/pdf2htmlEX/tmp
fi
if test -z "$PDF2HTMLEX_PREDIR" ; then
export PDF2HTMLEX_PREDIR=/tmp/pdf2htmlEX/pre
fi
if test -z "$PDF2HTMLEX_TEST_DIR" ; then
export PDF2HTMLEX_TEST_DIR=.
fi
# clear out the TMPDIR
#
rm -rf $PDF2HTMLEX_TMPDIR
mkdir -p $PDF2HTMLEX_TMPDIR
#
# setup the correct data files
#
cat $PDF2HTMLEX_TEST_DIR/../share/manifest | \
copy_TEST_IGNORE_file > $PDF2HTMLEX_DATDIR/manifest
cp $PDF2HTMLEX_TEST_DIR/../share/base.min.css $PDF2HTMLEX_DATDIR
cp $PDF2HTMLEX_TEST_DIR/../test/fancy.min.css $PDF2HTMLEX_DATDIR
runPdf2htmlEX 'test_fail.pdf'
runPdf2htmlEX 'basic_text.pdf'
runPdf2htmlEX 'geneve_1564.pdf'
runPdf2htmlEX 'text_visibility.pdf' '--correct-text-visibility=1'
runPdf2htmlEX 'with_form.pdf' '--process-form=1'
runPdf2htmlEX 'invalid_unicode_issue477.pdf'
runPdf2htmlEX 'svg_background_with_page_rotation_issue402.pdf' '--bg-format=svg'
runPdf2htmlEX 'fontfile3_opentype.pdf'
# clear out the PREDIR
#
rm -rf $PDF2HTMLEX_PREDIR
mkdir -p $PDF2HTMLEX_PREDIR
#
cp $PDF2HTMLEX_TMPDIR/* $PDF2HTMLEX_PREDIR

12
pdf2htmlEX/test/runLocalBrowserTests
View File

@ -1,8 +1,6 @@
#!/bin/bash
#!/bin/sh
set -ev
# This bash script runs the local browser tests
# This shell script runs the local browser tests
# We start by running a virtual frame buffer as display 99.0
#
@ -50,4 +48,8 @@ ps -ef | grep -v grep | grep Xvfb
echo "-----------------------------------------------------------"
echo ""
exit $returnCode
if [ -z "$TRAVIS_DIST"] ; then
exit $returnCode
fi
exit 0

7
pdf2htmlEX/test/runLocalTests
View File

@ -1,7 +0,0 @@
#!/bin/bash
set -ev
# This bash script runs the (simple non-browser) tests
python3 test_output.py

7
pdf2htmlEX/test/runLocalTestsPython Executable file
View File

@ -0,0 +1,7 @@
#!/bin/sh
set -ev
# This shell script runs the (simple non-browser) tests
python3 test_output.py

7
pdf2htmlEX/test/runLocalTestsShell Executable file
View File

@ -0,0 +1,7 @@
#!/bin/sh
set -ev
# This shell script runs the (simple non-browser) tests
./testOutput

17
pdf2htmlEX/test/test.py.in
View File

@ -12,9 +12,11 @@ class Common(object):
Variables and methods for common use in different tests.
"""
PDF2HTMLEX_PATH = "@PDF2HTMLEX_PATH@" # defined in CMakeLists.txt
if not os.path.isfile(PDF2HTMLEX_PATH) or not os.access(PDF2HTMLEX_PATH, os.X_OK):
print >> sys.stderr, "Cannot locate pdf2htmlEX executable, expected at ", PDF2HTMLEX_PATH,
". Make sure source was built before running this test."
PDF2HTMLEX_EXE = PDF2HTMLEX_PATH.split()
PDF2HTMLEX_EXE = PDF2HTMLEX_EXE[0]
if not os.path.isfile(PDF2HTMLEX_EXE) or not os.access(PDF2HTMLEX_EXE, os.X_OK):
print( sys.stderr, "Cannot locate pdf2htmlEX executable, expected at ", PDF2HTMLEX_EXE,
". Make sure source was built before running this test." )
exit(1)
SRC_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
@ -28,6 +30,8 @@ class Common(object):
PNGDIR = "@PDF2HTMLEX_PNGDIR@"
DATDIR = "@PDF2HTMLEX_DATDIR@"
OUTDIR = "@PDF2HTMLEX_OUTDIR@"
PREDIR = "@PDF2HTMLEX_PREDIR@"
HTMDIR = "@PDF2HTMLEX_HTMDIR@"
def setUp(self):
# filter manifest
@ -61,8 +65,11 @@ class Common(object):
shutil.rmtree(self.TMPDIR, ignore_errors=False, onerror=None)
os.mkdir(self.TMPDIR)
args = [Common.PDF2HTMLEX_PATH,
'--data-dir', self.DATDIR,
# To allow the user to specify additional command line arguments
# in the path, we split the PATH on whitespace and then append our
# arguments.
#
args = Common.PDF2HTMLEX_PATH.split() + [ '--data-dir', self.DATDIR,
'--dest-dir', self.TMPDIR
] + args

206
pdf2htmlEX/test/testOutput Executable file
View File

@ -0,0 +1,206 @@
#!/bin/sh
# This shell script checks that pdf2htmlEX does not crash, and produces
# correct files. Do not check the content of the files.
# We use a shell script to implement the python3 test_output.py so that we
# do not need to install any extra packages to test this functionality.
# This is how we run pdf2htmlEX on a particular file, and arguments.
#
runPdf2htmlEX() {
# collect the arguments
#
export LAST_TEST_NAME="$*"
#
pdfFileName=$1
#
shift
arguments=$*
# clear out the TMPDIR
#
rm -rf $PDF2HTMLEX_TMPDIR
mkdir -p $PDF2HTMLEX_TMPDIR
#
# now run pdf2htmlEX to produce the output files
#
echo "test: [$LAST_TEST_NAME]"
echo "---"
$PDF2HTMLEX_PATH \
--data-dir $PDF2HTMLEX_DATDIR \
--dest-dir $PDF2HTMLEX_TMPDIR \
$PDF2HTMLEX_TEST_DIR/test_output/$pdfFileName \
$arguments
}
# This is how we test for expected output files
#
hasExpectedFiles() {
filesFound="true"
for anExpectedFile in $1 ; do
if ! test -r $PDF2HTMLEX_TMPDIR/$anExpectedFile ; then
echo "NOT FOUND [$PDF2HTMLEX_TMPDIR/$anExpectedFile]"
filesFound="false"
fi
done
if test $filesFound = "true" ; then
echo "SUCCESS: $LAST_TEST_NAME" >> testOutputResults
else
echo "FAILURE: $LAST_TEST_NAME" >> testOutputResults
fi
echo "---"
echo ""
}
# This is how we copy a file omitting lines between '#TEST_IGNORE_BEGIN'
# and '#TEST_IGNORE_END' (we pipe the file in via stdin and save it via
# stdout)
#
copy_TEST_IGNORE_file() {
skipLine=echo
while IFS= read -r line ; do
if echo $line | grep -q "^#TEST_IGNORE_BEGIN" ; then
skipLine=true
elif echo $line | grep -q "^#TEST_IGNORE_END" ; then
skipLine=echo
else
$skipLine "$line"
fi
done
}
if test -z "$PDF2HTMLEX_PATH" ; then
echo "PANIC: we do not know where to find the pdf2htmlEX executable"
exit 1
fi
if test -z "$PDF2HTMLEX_DATDIR" ; then
export PDF2HTMLEX_DATDIR=/tmp/pdf2htmlex/dat
fi
mkdir -p $PDF2HTMLEX_DATDIR
if test -z "$PDF2HTMLEX_TMPDIR" ; then
export PDF2HTMLEX_TMPDIR=/tmp/pdf2htmlex/tmp
fi
mkdir -p $PDF2HTMLEX_TMPDIR
if test -z "$PDF2HTMLEX_TEST_DIR" ; then
export PDF2HTMLEX_TEST_DIR=.
fi
mkdir -p $PDF2HTMLEX_TEST_DIR
# Make sure any previous testOutputResults are cleared
#
rm -f testOutputResults
# setup the correct data files
#
cat $PDF2HTMLEX_TEST_DIR/../share/manifest | \
copy_TEST_IGNORE_file > $PDF2HTMLEX_DATDIR/manifest
cp $PDF2HTMLEX_TEST_DIR/../share/base.min.css $PDF2HTMLEX_DATDIR
cp $PDF2HTMLEX_TEST_DIR/../test/fancy.min.css $PDF2HTMLEX_DATDIR
# Do the tests
#
echo ""
echo "-------------------------------------------------------"
echo "running testOutput tests (simple non-browser tests)"
echo "-------------------------------------------------------"
echo ""
export LAST_TEST_NAME="unknown"
#
runPdf2htmlEX '1-page.pdf' --version
runPdf2htmlEX '1-page.pdf'
hasExpectedFiles '1-page.html'
runPdf2htmlEX '2-pages.pdf'
hasExpectedFiles '2-pages.html'
runPdf2htmlEX '1-page.pdf' 'foo.html'
hasExpectedFiles 'foo.html'
runPdf2htmlEX '2-pages.pdf' 'foo.html'
hasExpectedFiles 'foo.html'
runPdf2htmlEX '1-page.pdf' '--split-pages=1'
hasExpectedFiles '1-page.html 1-page1.page'
runPdf2htmlEX '3-pages.pdf' '--split-pages=1'
hasExpectedFiles '3-pages.html 3-pages1.page 3-pages2.page 3-pages3.page'
runPdf2htmlEX '1-page.pdf' '--split-pages=1 --page-filename=foo.xyz'
hasExpectedFiles '1-page.html foo1.xyz'
runPdf2htmlEX '3-pages.pdf' '--split-pages=1 --page-filename=foo.xyz'
hasExpectedFiles '3-pages.html foo1.xyz foo2.xyz foo3.xyz'
runPdf2htmlEX '3-pages.pdf' '--split-pages=1 --page-filename=fo%do.xyz'
hasExpectedFiles '3-pages.html fo1o.xyz fo2o.xyz fo3o.xyz'
runPdf2htmlEX '3-pages.pdf' '--split-pages=1 --page-filename=fo%03do.xyz'
hasExpectedFiles '3-pages.html fo001o.xyz fo002o.xyz fo003o.xyz'
runPdf2htmlEX '3-pages.pdf' '--split-pages=1 --page-filename=f%do%do.xyz'
hasExpectedFiles '3-pages.html f1o%do.xyz f2o%do.xyz f3o%do.xyz'
runPdf2htmlEX '3-pages.pdf' '--split-pages=1 --page-filename=f%soo.xyz'
hasExpectedFiles '3-pages.html f%soo1.xyz f%soo2.xyz f%soo3.xyz'
runPdf2htmlEX '3-pages.pdf' '--split-pages=1 --page-filename=f%poo.xyz'
hasExpectedFiles '3-pages.html f%poo1.xyz f%poo2.xyz f%poo3.xyz'
runPdf2htmlEX '3-pages.pdf' '--split-pages=1 --page-filename=f%noo.xyz'
hasExpectedFiles '3-pages.html f%noo1.xyz f%noo2.xyz f%noo3.xyz'
runPdf2htmlEX '3-pages.pdf' '--split-pages=1 --page-filename=f%%oo.xyz'
hasExpectedFiles '3-pages.html f%%oo1.xyz f%%oo2.xyz f%%oo3.xyz'
runPdf2htmlEX '3-pages.pdf' '--split-pages=1 --page-filename=f%%o%do.xyz'
hasExpectedFiles '3-pages.html f%%o1o.xyz f%%o2o.xyz f%%o3o.xyz'
runPdf2htmlEX '3-pages.pdf' '--split-pages=1 --page-filename=fo%do%%.xyz'
hasExpectedFiles '3-pages.html fo1o%%.xyz fo2o%%.xyz fo3o%%.xyz'
runPdf2htmlEX '3-pages.pdf' '--split-pages=1 --page-filename=f%02%doo.xyz'
hasExpectedFiles '3-pages.html f%021oo.xyz f%022oo.xyz f%023oo.xyz'
runPdf2htmlEX '1-page.pdf' '--split-pages=1 --page-filename=foo'
hasExpectedFiles '1-page.html foo1'
runPdf2htmlEX '2-pages.pdf' 'foo%d.html'
hasExpectedFiles 'foo%d.html'
runPdf2htmlEX '2-pages.pdf' 'foo%p.html'
hasExpectedFiles 'foo%p.html'
runPdf2htmlEX '2-pages.pdf' 'foo%n.html'
hasExpectedFiles 'foo%n.html'
runPdf2htmlEX '2-pages.pdf' 'foo%%.html'
hasExpectedFiles 'foo%%.html'
runPdf2htmlEX 'issue501' '--split-pages=1 --embed-css=0'
hasExpectedFiles ''
# Let the user know what the testOutputResults were
#
echo ""
echo "-------------------------------------------------------"
echo "testOutput results:"
echo "-------------------------------------------------------"
cat testOutputResults
echo "-------------------------------------------------------"
echo ""
# fail the script IF we find the work 'FAILURE' in the testOutputResults
#
if grep FAILURE testOutputResults ; then
exit 1
fi

5
pdf2htmlEX/test/test_local_browser.py
View File

@ -43,4 +43,7 @@ class test_local_browser(BrowserTests, unittest.TestCase):
self.browser.save_screenshot(png_file)
if __name__ == '__main__':
unittest.main()
# turn warnings OFF to prevent 'ResourceWarning: unclosed <socket.socket'
# see: https://stackoverflow.com/a/21500796
#
unittest.main(warnings='ignore')

36
testScripts/doItAll
View File

@ -1,36 +0,0 @@
#!/bin/bash
# This bash script automates the tests
mkdir -p testDir
cd testDir
wget https://github.com/stephengaito/pdf2htmlEX/releases/download/continuous/appImageName
export APPIMAGE_NAME=$(cat appImageName)
wget https://github.com/stephengaito/pdf2htmlEX/releases/download/continuous/$APPIMAGE_NAME
chmod a+x $APPIMAGE_NAME
if ! ./$APPIMAGE_NAME --version ; then
echo "COULD NOT run the AppImage"
exit 1
fi
if [ -x "$(which docker)" ]; then
wget https://github.com/stephengaito/pdf2htmlEX/releases/download/continuous/dockerImageName
export DOCKER_IMAGE_NAME=$(cat dockerImageName)
if ! docker run $DOCKER_IMAGE_NAME --version ; then
echo "COULD NOT run the Docker Image"
exit 1
fi
fi

32
testScripts/doItAllTravis
View File

@ -1,32 +0,0 @@
#!/bin/bash
# This bash script automates the tests
wget https://github.com/stephengaito/pdf2htmlEX/releases/download/continuous/appImageName
export APPIMAGE_NAME=$(cat appImageName)
wget https://github.com/stephengaito/pdf2htmlEX/releases/download/continuous/$APPIMAGE_NAME
chmod a+x $APPIMAGE_NAME
if ! ./$APPIMAGE_NAME --version ; then
echo "COULD NOT run the AppImage"
exit 1
fi
if [ -x "$(which docker)" ]; then
wget https://github.com/stephengaito/pdf2htmlEX/releases/download/continuous/dockerImageName
export DOCKER_IMAGE_NAME=$(cat dockerImageName)
if ! docker run $DOCKER_IMAGE_NAME --version ; then
echo "COULD NOT run the Docker Image"
exit 1
fi
fi

16
testScripts/runLocalTests
View File

@ -1,16 +0,0 @@
#!/bin/bash
set -ev
# This bash script runs the pdf2htmlEX tests
cd pdf2htmlEX/test
# The following is only needed for the local browser tests
#
./installAutomaticTestSoftware
./runLocalTests
./runLocalBrowserTests