refactored Alpine image creation

This commit is contained in:
Stephen Gaito 2020-06-21 12:00:27 +01:00
parent f340abb31e
commit 0d2c5048b0
29 changed files with 186 additions and 172 deletions

View File

@ -23,7 +23,7 @@ A few channels are available to reach the developers, please find the most prope
### The Issue Tracker
The [Issue Tracker](https://github.com/coolwanglu/pdf2htmlEX/issues)
The [Issue Tracker](https://github.com/pdf2htmlEX/pdf2htmlEX/issues)
is the best way for
[bug reports](#bug-reports),
[features requests](#feature-requests)
@ -54,7 +54,7 @@ However do not report issues or submit patches there, since it's terrible to kee
pdf2htmlEX is mostly written and maintained by 王璐 (Lu Wang).
His email and twitter account can be found in
[README.md](https://github.com/coolwanglu/pdf2htmlEX/blob/master/README.md).
[README.md](https://github.com/pdf2htmlEX/pdf2htmlEX/blob/master/README.md).
Please post only messages that do not fit into the above channels, otherwise
note that he no longer replies with _Please post your question to the mailing list_ or _Please file an issue at GitHub_, consider your message already replied.
@ -73,7 +73,7 @@ follow the following steps to get it resolved as fast as possible:
First of all, did you realize that your question might have been already answered in one of the following places?
- [pdf2htmlEX Wiki](https://github.com/coolwanglu/pdf2htmlEX/wiki)
- [pdf2htmlEX Wiki](https://github.com/pdf2htmlEX/pdf2htmlEX/wiki)
- The manpage (run `man pdf2htmlEX`)
- Old posts in the [mailing list](#the-mailing-list) or the [issue tracker](#the-issue-tracker)
- [Google](http://www.google.com/)
@ -93,7 +93,7 @@ Bugs should always be reported to [the Issue Tracker](#the-issue-tracker).
Before you report any bug:
- Use the latest git version of pdf2htmlEX, since the issue may have been already fixed.
- Search for previous issues (open or closed), to make sure that the issue has not been reported before.
- If pdf2htmlEX crashed, take a look at [this article](https://github.com/coolwanglu/pdf2htmlEX/wiki/Troubleshooting-Crashes).
- If pdf2htmlEX crashed, take a look at [this article](https://github.com/pdf2htmlEX/pdf2htmlEX/wiki/Troubleshooting-Crashes).
A good bug report shouldn't leave others needing to chase you up for more information.
The developers may be very familiar with the code base of pdf2htmlEX,
@ -110,7 +110,7 @@ If you are not sure, please try to answer the following questions:
- What steps will reproduce the issue? — please try to remove unnecessary steps
- What's the result and what did you expect? — e.g. you can post screenshots
- What error messages did you see?
- What's the affected PDF file and which pages are causing the issue? Create a pull request on [this repo](https://github.com/coolwanglu/pdf2htmlEX-testcase).
- What's the affected PDF file and which pages are causing the issue? Create a pull request on [this repo](https://github.com/pdf2htmlEX/pdf2htmlEX-testcase).
Especially for issues regarding building pdf2htmlEX:
- Which compiler are you using?

View File

@ -1,3 +1,3 @@
For instructions of building the source code, visit:
https://github.com/coolwanglu/pdf2htmlEX/wiki/Building
https://github.com/pdf2htmlEX/pdf2htmlEX/wiki/Building

View File

@ -1,4 +1,4 @@
pdf2htmlEX (https://github.com/coolwanglu/pdf2htmlEX)
pdf2htmlEX (https://github.com/pdf2htmlEX/pdf2htmlEX)
Copyright (c) 2012-2014 Lu Wang <coolwanglu@gmail.com> and other contributors
pdf2htmlEX, as a whole package, is licensed under GPLv3 (or any later

View File

@ -35,16 +35,6 @@ For most users, you probably really want to simply download one of the
[precompiled versions of
`pdf2htmlEX`](https://github.com/pdf2htmlEX/pdf2htmlEX/releases):
- [AppImage](https://appimage.org/) : Download, make executable, and
run...
This will work on most Linuxes, and most recent Windows 10.
- [Docker](https://www.docker.com/) Image from the [`pdf2htmlEX` Docker
hub](https://hub.docker.com/orgs/pdf2htmlex/repositories).
This will work on any machine with Docker installed.
- [Debian archive](https://en.wikipedia.org/wiki/Dpkg) : Download,
[apt](https://en.wikipedia.org/wiki/APT_(software)) install locally,
and run...
@ -55,15 +45,33 @@ For most users, you probably really want to simply download one of the
Experienced users of Linux, may be able to repackage the `*.deb` we
provide for use with their favourite package management tool.
- [AppImage](https://appimage.org/) : Download, make executable, and
run...
This will work on most Linuxes, and most recent Windows 10.
(It will not currently work on MacOS or Alpine based machines).
- [Docker](https://www.docker.com/) Image from the [`pdf2htmlEX` Docker
hub](https://hub.docker.com/orgs/pdf2htmlex/repositories).
This will work on any machine with Docker installed.
(Note: that *advanced* use of `pdf2htmlEX` requires careful attention to
the configuration of various tools, such as fontconfig, iconv and your
locally available fonts use by the poppler and fontforge libraries. The
docker images created by the pdf2htmlEX team might not be as well
configured for *your needs* as a docker created and configured by you)
### Building yourself
To build `pdf2htmlEX` on your own machine, inside the root directory of
a fresh clone of the
To build `pdf2htmlEX` on a Debian/Apt related machine, inside the root
directory of a fresh clone of the
[pdf2htmlEX/pdf2htmlEX](https://github.com/pdf2htmlEX/pdf2htmlEX)
repository, type:
```
./buildScripts/buildInstallLocally
./buildScripts/buildInstallLocallyApt
```
This will automatically install all required development tools and
@ -75,6 +83,10 @@ installing `pdf2htmlEX` into /usr/local/bin.
[Debian](https://www.debian.org/) based distribution. such as
[Ubuntu](https://ubuntu.com/), [Linux Mint](https://linuxmint.com/), etc.
**NOTE:** there is currently an *experimental* build script,
`./buildScripts/buildInstallLocallyAlpine`, for builds in Alpine
environments.
## The problem
To provide its full functionality, the `pdf2htmlEX` sources make direct
@ -126,7 +138,7 @@ Each script can be used individually to re-run a particular step if needed.
Typically, most users, will run one of the following "top-level" scripts:
1. **`buildInstallLocally`**
1. **`buildInstallLocallyApt`** (**`buildInstallLocallyAlpine`**)
This will automate:
@ -138,10 +150,16 @@ Typically, most users, will run one of the following "top-level" scripts:
3. compile and install `pdf2htmlEX`.
2. **`createImages`**
The `*Apt` script will build on any machine which uses the
`apt`/`apt-get` command.
Following a successful `buildInstallLocally`, this will create the
following images:
The `*Alpine` script will build on any machine which uses the
`apk` command (Alpine).
2. **`createImagesApt`** (**`createImagesAlpine`**)
Following a successful `buildInstallLocallyApt`, the `createImagesApt`
shell script will create the following images:
1. AppImage
@ -149,10 +167,18 @@ Typically, most users, will run one of the following "top-level" scripts:
3. Debian archive
Following a successful `buildInstallLocallyAlpine`, the
`createImagesAlpine` shell script will create the following images:
1. Alpine tar file
2. Docker image
3. **`runTests`**
Following a successful `buildInstallLocally`, this will run the various
'local' tests reporting errors as they occur.
Following a successful `buildInstallLocallyApt` (or
`buildInstallLocallyAlpine` ), the `runTests` shell script will run the
various 'local' tests reporting errors as they occur.
When run in [Travis-ci](https://travis-ci.org/), failing browser tests
will *not* fail the overall Travis build, but will instead upload the
@ -192,6 +218,10 @@ Typically, most users, will run one of the following "top-level" scripts:
ensures that any versions of Poppler already installed by the user, are
not broken by the user's installation of `pdf2htmlEX`.
- **`createAlpineTarFile`**: Using an already compiled version of `pdf2htmlEX`,
installs it and `popper-data` into a tar file suitable for use in any
Alpine environment.
- **`createAppImage`**: Using an already compiled version of `pdf2htmlEX`,
installs it and `popper-data` into an AppImage.
@ -199,15 +229,28 @@ Typically, most users, will run one of the following "top-level" scripts:
`pdf2htmlEX`, installs it and `poppler-data` into a Debian archive
(`*.deb`).
- **`createDockerAlpineImageFromTarFile`**: Installs the Alpine tar file
archive of `pdf2htmlEX` created by `createAlpineTarFile` into an Alpine
Docker image.
- **`createDockerUbuntuImageFromDeb`**: Installs the Debian archive of
`pdf2htmlEX` created by `createDebianPackage` into a Docker image.
- **`getBuildTools`**: Locally `apt` installs all development *tools*
- **`getBuildToolsAlpine`**: Locally `apk` installs all development
*tools* required to build `pdf2htmlEX`.
- **`getBuildToolsApt`**: Locally `apt` installs all development *tools*
required to build `pdf2htmlEX`.
- **`getDevLibraries`**: Locally `apt` installs all development
- **`getDevLibrariesAlpine`**: Locally `apk` installs all development
*libraries* required to build `pdf2htmlEX`.
- **`getDevLibrariesApt`**: Locally `apt` installs all development
*libraries* required to build `pdf2htmlEX`.
This script provides a definitive list of all libraries required to run
`pdf2htmlEX`.
This script provides a definitive list of all libraries required to run
`pdf2htmlEX`.
@ -236,7 +279,7 @@ Typically, most users, will run one of the following "top-level" scripts:
The `PDF2HTMLEX_PREFIX` variable is specified in the `versionEnvs`
script.
- **`rutTests`**: Runs the tests located in the
- **`runTests`**: Runs the tests located in the
`pdf2htmlEX/pdf2htmlEX/test` directory. See the
`pdf2htmlEX/pdf2htmlEx/test` directory's Readme file for details.
@ -279,10 +322,6 @@ Typically, most users, will run one of the following "top-level" scripts:
used by the `uploadGitHubRelease` script as the contents of the release
message, as visible to the user, for the 'continuous' release section.
- **`dockerFunctions`**: A collection of shell functions used by the
`uploadDockerImage` script to automate the upload of the Docker images
to Docker Hub.
- **`listFilesByChangeTime`**: A simple shell script which lists the files
in the buildScripts directory by most recently changed files first.

View File

@ -47,11 +47,3 @@ set -ev
./buildScripts/buildPdf2htmlEX
./buildScripts/installPdf2htmlEX
#./buildScripts/createAppImage
#./buildScripts/createDockerUbuntuImageFromDeb
#./buildScripts/uploadGitHubRelease
#./buildScripts/uploadDockerImage

View File

@ -47,11 +47,3 @@ set -ev
./buildScripts/buildPdf2htmlEX
./buildScripts/installPdf2htmlEX
#./buildScripts/createAppImage
#./buildScripts/createDockerUbuntuImageFromDeb
#./buildScripts/uploadGitHubRelease
#./buildScripts/uploadDockerImage

View File

@ -0,0 +1,79 @@
#!/bin/sh
# This shell script creates a docker image from an existing pdf2htmlEX
#
# This is the part which can run *inside* a docker container.
echo ""
echo "-------------------------------------------------------------------"
echo "CREATING pdf2htmlEX Docker Image (run inside a docker container)"
echo " (based on Alpine linux $BUILD_DIST)"
echo "-------------------------------------------------------------------"
echo ""
# Collect everything that will be needed...
# source buildScripts/reSourceVersionEnvs
. buildScripts/reSourceVersionEnvs
export ALPINE_NAME="pdf2htmlEX-$PDF2HTMLEX_NAME"
echo "export ALPINE_NAME=\"$ALPINE_NAME\"" >> buildScripts/reSourceVersionEnvs
set -ev
mkdir -p imageBuild/alpineTarDir
cd pdf2htmlEX/build
sudo rm -rf install_manifest.txt
make install DESTDIR=../../imageBuild/alpineTarDir
cd ../../poppler-data
make install \
prefix=$PDF2HTMLEX_PREFIX \
datadir=$PDF2HTMLEX_PREFIX/share/pdf2htmlEX \
DESTDIR=../imageBuild/alpineTarDir
cd ../imageBuild
tar czvf $ALPINE_TARFILE_NAME.tar.gz alpineTarDir
cat <<ALPINE_INSTALL > $ALPINE_NAME.install
#!/bin/sh
# This (alpine) shell script installs a locally existing tar.gz file of
# the pdf2htmlEX binaries compiled on Alpine $DIST.
#
# You MUST have root/sudo privileges to run this file.
#
# We start by installing all of the (known) required runtime dependencies
#
apk update
#
apk add --no-cache \
tar \
libstdc++ \
libgcc \
gnu-libiconv \
gettext \
glib \
freetype \
fontconfig \
cairo \
libpng \
libjpeg-turbo \
libxml2
# Now we install the (Alpine $DIST) compiled pdf2htmlEX binaries and
# configuration files.
#
cd /
#
tar xvf $ALPINE_NAME.tar.gz --strip-comonents=1
ALPINE_INSTALL

View File

@ -16,15 +16,14 @@ echo ""
# source buildScripts/reSourceVersionEnvs
. buildScripts/reSourceVersionEnvs
# source buildScripts/dockerFunctions
. buildScripts/dockerFunctions
set -ev
mkdir -p imageBuild/dockerDir
cd imageBuild/dockerDir
cp ../$ALPINE_NAME.* .
if [ -z "$DOCKER_FROM" ]; then
echo ""
read -p "Enter the docker image for the 'from' base: " DOCKER_FROM
@ -54,21 +53,9 @@ echo "export DOCKER_NAME=\"$DOCKER_NAME\"" >> ../../buildScripts/reSourc
cat > Dockerfile <<DOCKERFILE_HERE_DOC
FROM $DOCKER_FROM
COPY ./ /
COPY ./$ALPINE_NAME.* /root
RUN apk update && \
apk add --no-cache \
libstdc++ \
libgcc \
gnu-libiconv \
gettext \
glib \
freetype \
fontconfig \
cairo \
libpng \
libjpeg-turbo \
libxml2
RUN /bin/sh -c /root/$ALPINE_NAME.install
ENTRYPOINT ["$PDF2HTMLEX_PREFIX/bin/pdf2htmlEX"]
DOCKERFILE_HERE_DOC

View File

@ -1,37 +0,0 @@
#!/bin/sh
# This shell script creates a docker image from an existing pdf2htmlEX
#
# This is the part which can run *inside* a docker container.
echo ""
echo "-------------------------------------------------------------------"
echo "CREATING pdf2htmlEX Docker Image (run inside a docker container)"
echo " (based on Alpine linux $BUILD_DIST)"
echo "-------------------------------------------------------------------"
echo ""
# Collect everything that will be needed...
# source buildScripts/reSourceVersionEnvs
. buildScripts/reSourceVersionEnvs
# source buildScripts/dockerFunctions
. buildScripts/dockerFunctions
set -ev
mkdir -p imageBuild/dockerDir
cd pdf2htmlEX/build
sudo rm -rf install_manifest.txt
make install DESTDIR=../../imageBuild/dockerDir
cd ../../poppler-data
make install \
prefix=$PDF2HTMLEX_PREFIX \
datadir=$PDF2HTMLEX_PREFIX/share/pdf2htmlEX \
DESTDIR=../imageBuild/dockerDir

View File

@ -15,9 +15,6 @@ echo ""
# source buildScripts/reSourceVersionEnvs
. buildScripts/reSourceVersionEnvs
# source buildScripts/dockerFunctions
. buildScripts/dockerFunctions
set -ev
cd imageBuild
@ -56,12 +53,14 @@ echo "export DOCKER_NAME=\"$DOCKER_NAME\"" >> ../../buildScripts/reSourc
cat > Dockerfile <<DOCKERFILE_HERE_DOC
FROM $DOCKER_FROM
COPY ./$DPKG_NAME /root
RUN apt update && \
apt -y upgrade && \
apt -y --no-install-recommends install \
/root/$DPKG_NAME
ENTRYPOINT ["$PDF2HTMLEX_PREFIX/bin/pdf2htmlEX"]
DOCKERFILE_HERE_DOC

19
buildScripts/createImagesAlpine Executable file
View File

@ -0,0 +1,19 @@
#!/bin/sh
set -ev
# This shell script creates the pdf2htmlEX AppImage and Docker Images
#################
# do the creation
./buildScripts/reportEnvs
./buildScripts/createAlpineTarFile
if [ -x "$(which docker)" ]; then
./buildScripts/createDockerAlpineImageFromTarFile
fi

View File

@ -1,59 +0,0 @@
# The following has been extracted and modified for our use under an MIT
# license by Stephen Gaito on 2019/NOV/25
#
# The original was taken from:
# https://raw.githubusercontent.com/AppImage/pkg2appimage/master/functions.sh
# (commit ID ed1d385282a6aa6c9a93b52296f20555adf9bae7 commited 2019/Apr/9)
#
# The original copyright was: Copyright (c) 2004-16 Simon Peter
#
# Copy the library dependencies of all exectuable files in the current
# directory (it can be beneficial to run this multiple times)
#
copy_deps()
{
PWD=$(readlink -f .)
FILES=$(find . -type f -executable -or -name *.so.* -or -name *.so \
| sort | uniq )
for FILE in $FILES ; do
ldd "${FILE}" | grep "=>" | awk '{print $3}' \
| xargs -I '{}' echo '{}' >> DEPSFILE
done
DEPS=$(cat DEPSFILE | sort | uniq)
for FILE in $DEPS ; do
if [ -e $FILE ] && [[ $(readlink -f $FILE)/ != $PWD/* ]] ; then
cp -v --parents -rfL $FILE ./ || true
fi
done
rm -f DEPSFILE
}
# Delete blacklisted files. Our blacklisted files are the shared libraries
# extracted from the "base" docker image
#
delete_blacklisted()
{
docker run --rm -it --entrypoint=find $DOCKER_FROM -iname "*.so.*" \
| xargs -i basename {} | sed 's/\r//g' | sed 's/\(so\.[0-9]\+\).*$/\1/' \
| sort | uniq > BLACKLISTED_FILES
for FILE in $(cat BLACKLISTED_FILES | tr "\r\n" " ") ; do
FILES="$(find . -name "${FILE}*" -not -path "./usr/optional/*")"
for FOUND in $FILES ; do
rm -vf "$FOUND" "$(readlink -f "$FOUND")"
done
done
rm BLACKLISTED_FILES
# Do not bundle developer stuff
rm -rf usr/include || true
rm -rf usr/lib/cmake || true
rm -rf usr/lib/pkgconfig || true
find . -name '*.la' | xargs -i rm {}
}

View File

@ -359,7 +359,7 @@ pdf2htmlEX is written by Lu Wang <coolwanglu@gmail.com>
.SH SEE ALSO
.TP
Home page
https://github.com/coolwanglu/pdf2htmlEX
https://github.com/pdf2htmlEX/pdf2htmlEX
.TP
pdf2htmlEX Wiki
https://github.com/coolwanglu/pdf2htmlEX/wiki
https://github.com/pdf2htmlEX/pdf2htmlEX/wiki

View File

@ -2,7 +2,7 @@
/*!
* Base CSS for pdf2htmlEX
* Copyright 2012,2013 Lu Wang <coolwanglu@gmail.com>
* https://github.com/coolwanglu/pdf2htmlEX/blob/master/share/LICENSE
* https://github.com/pdf2htmlEX/pdf2htmlEX/blob/master/share/LICENSE
*/
/* Part 1: Web Page Layout: Free to modify, except for a few of them which are required by pdf2htmlEX.js, see the comments */
#sidebar { /* Sidebar */

View File

@ -2,7 +2,7 @@
/*!
* Fancy styles for pdf2htmlEX
* Copyright 2012,2013 Lu Wang <coolwanglu@gmail.com>
* https://github.com/coolwanglu/pdf2htmlEX/blob/master/share/LICENSE
* https://github.com/pdf2htmlEX/pdf2htmlEX/blob/master/share/LICENSE
*/
@keyframes fadein { from { opacity:0;} to { opacity:1;} }
@-webkit-keyframes fadein { from { opacity:0;} to { opacity:1;} }

View File

@ -17,7 +17,7 @@
# Declaration - Do not modify
"""
<!DOCTYPE html>
<!-- Created by pdf2htmlEX (https://github.com/coolwanglu/pdf2htmlex) -->
<!-- Created by pdf2htmlEX (https://github.com/pdf2htmlEX/pdf2htmlEX) -->
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta charset="utf-8"/>

View File

@ -2,7 +2,7 @@
/**
* @license pdf2htmlEX.js: Core UI functions for pdf2htmlEX
* Copyright 2012,2013 Lu Wang <coolwanglu@gmail.com> and other contributors
* https://github.com/coolwanglu/pdf2htmlEX/blob/master/share/LICENSE
* https://github.com/pdf2htmlEX/pdf2htmlEX/blob/master/share/LICENSE
*/
/*

View File

@ -43,4 +43,7 @@ class test_local_browser(BrowserTests, unittest.TestCase):
self.browser.save_screenshot(png_file)
if __name__ == '__main__':
unittest.main()
# turn warnings OFF to prevent 'ResourceWarning: unclosed <socket.socket'
# see: https://stackoverflow.com/a/21500796
#
unittest.main(warnings='ignore')