Difference between revisions of "CGI"
m (→A script to upload a copy of a base)
m (→remote.sh for images or src update)
|Line 361:||Line 361:|
Revision as of 19:28, 25 October 2015
|Language:||English • français|
A shared hosting account have some restrictions :
- Daemon mode is forbidden, or cannot be activated. So you must use CGI mode.
- You must fit in the server environment : OS version, libraries, installed packages.
- The OS version is generally a stable version, but not the latest version.
Download a GeneWeb package
- File:btn lin.png made for recent Linux distribution
- Geneweb distribution made with Linux Debian Sarge, and currently running on 1&1 hosting.
- 1 Directories and files
- 2 Parameters of the CGI script
- 3 An alternative installation on 1&1
Directories and files
The following applies to 1&1 hosting, and may differ on other hostings: Under the root directory where CGI scripts may run, we have : !File:captur477.jpg
The directories :
|?||Beside the CGI-root directory|
|mybases||The bases directory|
|?||In the CGI-root directory|
|basesxg||An alternative bases directory|
|css||A copy of the gw/css directory. This directory is used by the Apache server.|
|gw||The gw directory of the Geneweb distribution
No change, except give the "exec" permission on the files gw/gwd, gw/gwsetup, and files used by gwsetup (gw/gwc*, gw/gwu,gw/consang, gw/update_nldb) The gwd.arg file is *empty*.
|gwenv||The gw directory of the Geneweb-5 distribution
On this web site, we can easily switch between the 2 last versions of GeneWeb.
|igw||The images directory for Geneweb-5|
|images||A directory where copies of gw/images/gwback.jpg, gw/images/gwlogo_bas.png|
|pub||A directory where are readable copies of the CGI scripts (this website is a demo site) :|
And the files :
|issue6.cgi||A test script which
- displays information about the environment in the server - checks the gwd binary file : size and md5sum
|issue5.cgi||The same script, but for the GeneWeb-5 version|
|gw6.cgi||The CGI script which launches GenWeb-6|
|gw5.cgi||The CGI script which launches GenWeb-5|
Parameters of the CGI script
PWD=$(pwd) LNG="fr" GENEWEBSHARE=$PWD/gw GENEWEBDOC=$PWD/gw/doc GENEWEBDB=$PWD/../bases DAEMON=$GENEWEBSHARE/gwd LOGFILE=$GENEWEBDB/gw.log
Main settings :
- 1 get the CGI working directory
- 2-7 set the main parameters
- 5 $GENEWEBDB does not need to be accessed by the HTTP server.
It must be protected either by a .htaccess file, either by a location out of the HTTP server scope.
- 7 a log file is useful when there are problems, but do not forget to delete it after downloading.
OPTIONS="-blang -robot_xcl 40,70 -max_clients 15 -conn_tmout 120 -min_disp_req 30 -images_url http://myserver.net/gw/images" # -allowed_tags $GENEWEBDB/tags.txt cd $GENEWEBSHARE $DAEMON -hd$GENEWEBSHARE -dd$GENEWEBDOC -bd$GENEWEBDB -lang$LNG -log$LOGFILE -cgi $OPTIONS 2>/dev/null
Options and launching
- 1 OPTIONS
|-robot_xcl||to protect your data from HTTrack or WebSite Extractor|
|-conn_tmout||for statistics on the bottom line|
|-images_url||icons and images are not sent by GeneWeb, but par your HTTP server (not CGI).|
|-allowed_tags||another useful option, if you use HTML tags not in default_good_tag_list|
- 3 cd $GENEWEBSHARE. I am not sure it is really needed.
- 4 launch GeneWeb
An alternative installation on 1&1
Some general comments
Installing GeneWeb on a remote shared server is a little bit (but not that much) more complex than on a personal computer. GeneWeb is a totally self contained package, therefore does not depend on additional libraires. On a server, GeneWeb cannot execute in daemon mode (Unix aficionados will understand), and must be launched in CGI mode. To that extent, a server based installation requires prior installation of a (standard) HTTP server (typically Apache), access to the CGI repository of this server and some control over the access rights to the various folders and files. Installing or verifying your HTTP server is not part of this discussion.
A second restriction of GeneWeb on a server is that gwsetup cannot be used (mainly for security reasons). It is therefore necessary to use command lines on the remote server through a
ssh window or any other remote access tool of your choice.
It is also necessary to have access to a upload mechanism from your personal machine to the server through classical Unix/Linux command lines, or through a
ftp client such as Filezilla. A section below will describe in detail a shell script performing upload to the server of a complete fresh copy of a base sitting on your personal computer.
Folders and files
The first important file for your installation is the welcome page of your HTTP server, by default
index.html sitting at the root of your server. After personalization of this welcome page, you should verify that you can view it by typing the name (or IP address) of your server in the URL window of your browser.
The second important item is the CGI folder containing the executable files invoqued through URLs.
This folder is typically called
cgi-bin and also sits at the root of your HTTP server.
If this folder contains an executable file named
test-cgi.sh as shown below:
(uiserver):u723:~ > cd cgi-bin (uiserver):u723:~/cgi-bin > cat test-cgi.sh #!/bin/sh echo 'Content-type: text/htm' echo echo '<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" ' echo ' "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd"> ' echo '<html xmlns="http://www.w3.org/1999/xhtml"> ' echo '<body>' echo 'This is a test for cgi commands' echo '</body>' echo '</html>' (uiserver):u723:~/cgi-bin > chmod 755 test-cgi.sh # turn this file into executable (uiserver):u723:~/cgi-bin >./test-cgi.sh Content-type: text/htm' <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.1//EN" "http://www.w3.org/TR/xhtml11/DTD/xhtml11.dtd"> <html xmlns="http://www.w3.org/1999/xhtml"> <body> This is a test for cgi commands </body> </html> (uiserver):u723:~/cgi-bin >
then typing in the URL window of your browser the following text
should result in the text This is a test for cgi commands appearing in your browser window (provided you have done a
chmod 755 test-cgi.sh on the remote server).
You must reach this level of functionality to proceed.
At the root of your server, create two additional folders
- The first folder will contain all GeneWeb related files (binaries, bases, images, ...) and need not or should not be accessible with the HTTP server,
- The second folder will contain any other documents that you may want to share with the visitors of your site.
At this time, we have the following situation:
(uiserver):u72332903:~ > cd (uiserver):u72332903:~ > ls -al total 554088 drwx---r-t 23 u72332903 ftpusers 4096 Oct 7 06:58 . drwxr-xr-t 7 root root 4096 Oct 23 00:21 .. -rw------- 1 u72332903 ftpusers 8319 Oct 10 09:30 .bash_history drwxr-xr-x 3 u72332903 ftpusers 4096 Sep 24 21:24 Documents drwxr-xr-x 10 u72332903 ftpusers 4096 Sep 24 21:24 GeneWeb drwxr-xr-x 3 u72332903 ftpusers 4096 Oct 7 06:58 cgi-bin -rw-r--r-- 1 u72332903 ftpusers 3535 May 22 14:30 index.html -rw-r--r-- 1 u72332903 ftpusers 27 Nov 6 2013 robots.txt (uiserver):u72332903:~ >
The GeneWeb folder
The GeneWeb folder contains one or more versions of GeneWeb (
geneweb-7.00 in our example), and the
geneweb-x.yy folder has the same structure, containing a copy of the
distribution folder resulting from the compilation of GeneWeb. You can upload with
ftp this folder, or you can compile yourself GeneWeb directly on your server, but this necessitates the installation of Ocaml and camlp5 (see Ocaml).
(uiserver):u72332903:~/GeneWeb > cd geneweb-6.08 (uiserver):u72332903:~/GeneWeb/geneweb-6.08 > ls -al total 204 drwx---r-x 3 u72332903 ftpusers 82 Jan 12 2015 . drwxr-xr-x 10 u72332903 ftpusers 4096 Sep 24 21:24 .. -rw----r-- 1 u72332903 ftpusers 160302 Jan 12 2015 CHANGES.txt -rw----r-- 1 u72332903 ftpusers 18007 Jan 12 2015 LICENSE.txt -rw----r-- 1 u72332903 ftpusers 10345 Jan 12 2015 START.htm drwx---r-x 8 u72332903 ftpusers 4096 Dec 17 2014 gw -rw---r-x 1 u72332903 ftpusers 108 Dec 17 2014 gwd.command -rw---r-x 1 u72332903 ftpusers 1117 Dec 17 2014 gwsetup.command (uiserver):u72332903:~/GeneWeb/geneweb-6.08 > (uiserver):u72332903:~/GeneWeb/geneweb-6.08 > cd gw (uiserver):u72332903:~/GeneWeb/geneweb-6.08/gw > ls -al total 13268 drwx---r-x 8 u72332903 ftpusers 4096 Dec 17 2014 . drwx---r-x 3 u72332903 ftpusers 82 Jan 12 2015 .. -rw----r-- 1 u72332903 ftpusers 15280 Jan 12 2015 a.gwf -rwx---r-x 1 u72332903 ftpusers 529764 Jan 12 2015 consang drwx---r-x 12 u72332903 ftpusers 4096 Dec 2 2014 etc -rwx---r-x 1 u72332903 ftpusers 1182404 Jan 12 2015 ged2gwb -rwx---r-x 1 u72332903 ftpusers 1200900 Jan 12 2015 ged2gwb2 -rwx---r-x 1 u72332903 ftpusers 552100 Jan 12 2015 gwb2ged -rwx---r-x 1 u72332903 ftpusers 631844 Jan 12 2015 gwc -rwx---r-x 1 u72332903 ftpusers 861434 Jan 12 2015 gwc1 -rwx---r-x 1 u72332903 ftpusers 630084 Jan 12 2015 gwc2 -rwx---r-x 1 u72332903 ftpusers 2422372 Jan 12 2015 gwd -rw----r-- 1 u72332903 ftpusers 12 Jan 12 2015 gwd.arg -rw----r-- 1 u72332903 ftpusers 0 Jan 12 2015 gwd.log -rw----r-- 1 u72332903 ftpusers 42 Oct 10 2014 gwd.xcl -rw----r-- 1 u72332903 ftpusers 3851203 Jan 12 2015 gwdl.log -rwx---r-x 1 u72332903 ftpusers 358308 Jan 12 2015 gwsetup drwx---r-x 3 u72332903 ftpusers 68 Jan 3 2015 gwtp_tmp -rwx---r-x 1 u72332903 ftpusers 589028 Jan 12 2015 gwu drwx---r-x 3 u72332903 ftpusers 4096 Nov 20 2014 images drwx---r-x 2 u72332903 ftpusers 140 Jan 12 2015 lang drwx---r-x 2 u72332903 ftpusers 6 Sep 24 2014 old -rw----r-- 1 u72332903 ftpusers 10 Jan 12 2015 only.txt drwx---r-x 3 u72332903 ftpusers 33 Sep 24 2014 setup -rw----r-- 1 u72332903 ftpusers 104 Dec 17 2014 tags.txt -rwx---r-x 1 u72332903 ftpusers 497028 Jan 12 2015 update_nldb (uiserver):u72332903:~/GeneWeb/geneweb-6.08/gw >
The cgi folder
cgi-bin folder contains the cgi commands launching the gwd server software for each request from a browser client.
(uiserver):u72332903:~ > cd cgi-bin/ (uiserver):u72332903:~/cgi-bin > ls -al total 144 drwxr-xr-x 3 u72332903 ftpusers 4096 Oct 7 06:58 . drwx---r-t 23 u72332903 ftpusers 4096 Oct 7 06:58 .. lrwxrwxrwx 1 u72332903 ftpusers 8 Jan 12 2015 Gwd -> Gwd-7.00 -rwxr-xr-x 1 u72332903 ftpusers 183 Oct 10 2014 Gwd-6.08 -rwx---r-x 1 u72332903 ftpusers 183 Jan 12 2015 Gwd-7.00 -rwx---r-x 1 u72332903 ftpusers 183 Jan 15 2015 test-cgi.sh (uiserver):u72332903:~/cgi-bin >
Gwd is a symbolic link to the current version of GeneWeb (7.00 in our example).
Each of the
gwd-x.yy file is a shell script that launches
gwd with the appropriate parameters:
#!/bin/sh DIR=~/GeneWeb/geneweb-6.08/gw BASE_DIR=~/GeneWeb/bases OPTIONS="-robot_xcl 19,60 -allowed_tags ./tags.txt -hd ./" cd $DIR ./gwd -cgi $OPTIONS -bd ../../bases > ./gwd.log 2>&1
To change your server to another version of GeneWeb (7.00 for instance), you should do the following:
cd cgi-bin rm -f Gwd ln -s Gwd-7.00 Gwd
The bases folder
The base folder should contain the same information structure as in the case of installation on a personal computer:
- one or several base.gwb folders.
- one or several base.gwf files.
- cnt containing access count data.
- etc containing template text files used in priority over the generic
gw/etcfiles (see header).
- images containing for each base photos associated with each person (
- lang containing some base specific language relates template text files (lexicon for instance).
- src containing for each base text files and image files inserted in notes with the m=SRC command (see gwd commands).
- wiz.auth as many wizards and friends authorization files as specified for each base in their respective .gwf parameter file.
(Note that the picture on the right has been taken off a personnal computer where the bases are stored in a folder names
GeneWeb-Bases, as opposed to
bases in the remote server structure proposed here).
A script to upload a copy of a base
The script in this page takes a
base_name as an argument, and optionally
src. It uploads a fresh base on your server, or a fresh copy of the
src/base folders. It performs the following steps:
- creates a fresh base.gw file from the base on your personal computer.
- extracts a listing of the content of
- creates a
remote.shfile (see below).
- creates a tar file containing $BASE.gw, history remote.sh and the three
- sends the tar file to the remote server.
- triggers on the remote server unfolding of the tar file and execution of
remote.sh. The result of this remote execution should be a mail containing the
remote.logand three diff between images folders.
This procedure saves the previous folder of the base or its images and src folders with a "yy-mm-dd" date tag. As time progresses, you may want to clean-up this accumulation of saves.
remote.sh for base update
The shell script below is executed on your remote server, and installs your base at the proper location (according to the overall set-up described here).
#!/bin/sh # BASE will be replaced by the appropriate name by sed cd DATE=$(date +"%Y-%m-%d-%T") echo 'Error log of update for base: BASE' > ./remote.log echo $DATE >> ./remote.log # Save current version cd ~/GeneWeb/bases 2>> ./remote.log if [ -d ./BASE.gwb ] # do it only if folder exists then mv ./BASE.gwb ./BASE-$DATE.gwb 2>> ./remote.log fi # Extract new base from .tar file tar xf ~/BASE.tar 2>> ./remote.log # rebuild BASE from .gw file ../geneweb-VERS_S/gw/gwc1 -f -o BASE BASE.gw 2>> ./remote.log ../geneweb-VERS_S/gw/updnldb BASE 2>> ./remote.log if [ -f ./BASE.gwb ] # do it only if file exists then cp -f ./BASE-$DATE.gwb/history ./BASE.gwb 2>> ./remote.log fi if [ -f ./BASE.gwb ] # do it only if file exists then cp -f ./BASE-$DATE.gwb/forum ./BASE.gwb 2>> ./remote.log fi # done cd >> ./remote.log # Create a link to css.txt in the bases/src folder ln -s -f ~/GeneWeb/geneweb-$VERS/gw/etc/css.txt ~/GeneWeb/bases/src/BASE/css.txt 2>> ./remote.log # compute diffs between server and personal computer for image folders if [ -d ~/GeneWeb/bases/images/BASE ] # do it only if folder exists then ls ~/GeneWeb/bases/images/BASE > ./ls-personnes-serveur.txt fi if [ -d ~/GeneWeb/bases/src/BASE/ ] # do it only if folder exists then ls ~/GeneWeb/bases/src/BASE/*.txt > ./ls-src-files-serveur.txt fi if [ -d ~/GeneWeb/bases/src/BASE/images ] # do it only if folder exists then ls ~/GeneWeb/bases/src/BASE/images > ./ls-images-serveur.txt fi echo 'Personnes diff serveur vs local' >> ./remote.log diff ./ls-personnes-serveur.txt ./ls-personnes.txt >> ./remote.log echo 'Src files diff serveur vs local' >> ./remote.log diff ./ls-src-files-serveur.txt ./ls-src-files.txt >> ./remote.log echo 'Images diff serveur vs local' >> ./remote.log diff ./ls-images-serveur.txt ./ls-images.txt >> ./remote.log /usr/bin/mail -s 'Update error log' 'ADDRESS' < ./remote.log rm -f ./remote.*
remote.sh for images or src update
The shell script below is executed on your remote server, and installs the
src/images folders for your base at the proper location (according to the overall set-up described here).
Depending on the size of your images folders, and on the number of new images requiring upload, you may prefer to upload images individually with your preferred
ftp client rather than doing the bulk upload proposed here. Remember that the base upload script performs a comparison between the images folders on your personal computer and your server.
#!/bin/sh # BASE and ADDRESS will be replaced by the appropriate values by sed DATE=$(date +"%Y-%m-%d-%T") if [ -e ./images.tmp ] then FILES="images" rm -f ./images.tmp fi if [ -e ./src.tmp ] then FILES="src" rm -f ./src.tmp fi echo "Error log of update for $FILES of base: BASE" > ./remote.log echo "`date`" >> ./remote.log # Save current version then move new version if [ -d ~/GeneWeb/bases/$FILES/BASE ] # do it only if folder exists then mv ~/GeneWeb/bases/$FILES/BASE ~/GeneWeb/bases/$FILES/BASE-$DATE 2>> ./remote.log fi mv ./BASE ~/GeneWeb/bases/$FILES 2>> ./remote.log /usr/bin/mail -s 'Update error log' 'ADDRESS' < ./remote.log rm -f ./remote.*
Debugging your remote server
Debugging your remote server may be tricky, but some systematic approach and several tools will help.
- Verify first that your HTTP server works properly. This is achieved by typing
your_server_namein the URL window of your browser which should return the content of
- Verify that the cgi mechanism works. This is achieved by typing
your_server_name/cgi-bin/test-cgi.shas seen above (see #Folders and files).
- If your server returns a "Internal server error", several hypothesis should be examined:
- Your script test-cgi.sh does not work properly. Run it directly on a terminal window by typing
./test-cgi.sh(see #Folders and files).
- The first two lines returned by your test-cgi.sh script are not exactly as shown (the second one should be a blank line).
- Examine the HTTP server log file on your server by typing:
- (your environment may store access logs somewhere else, but this is the most likely place)
- Examination of the log file may give you a hint as to the nature of your problem.
- Your script test-cgi.sh does not work properly. Run it directly on a terminal window by typing
- Another typical issue is that of ownership of the various files associated with GeneWeb. In the case described here, ownership is
userand group ownership is
ftpusers. Depending on the specific method you have used to install GeneWeb, this may vary. Some discussions on the Yahoo! group forum about GeneWeb mentions geneweb for group ownership!
- Lastly, you may want to look at the gwd log file if your GeneWeb server works only partially (welcome page is ok, but other pages do not work properly). This log file is specified in the gwd launch command. In our example, it sits at
where $DIR depends on the specific version of GeneWeb you are currently running (see details in #The cgi folder section above).