Authentication in Apache

I wanted to restrict access to a directory on my web server. It’s quite easy by adding an Authentication directive in the site’s configuration file (in /etc/apache2/sites-available/; preferred) or in a .htaccess file in the directory itself.

Create the user and password:

<pre class="src src-sh">htpasswd -c /path/to/my/specified/password/file user.name <span style="color: #ff4500;">## </span><span style="color: #ff4500;">place file to a place that is not accessible on the web, maybe where htdocs is located.</span>

## enter password

In the site’s configuration file, add a directory directive and add in Authentication. It should look something like:

<pre class="src src-sh">AuthType Basic

AuthName “Restricted Files” # (Following line optional) AuthBasicProvider file AuthUserFile /path/to/my/specified/password/file Require user user.name

For many people, I can use groups.

Pretty easy. Note that the user will be able to access that directory from the browser until the browser is closed.

SSL in Apache

I recently tested setting up SSL for my web server. I will outline how I set this up using a self-signed certificate. Some useful references are this, this, this, and this.

I assume Apache is up and running and OpenSSL is installed.

Set up SSL certificates:

<pre class="src src-sh">sudo a2enmod ssl

cd /etc/apache2 sudo openssl genrsa -des3 -out server.key 1024 ## leave out -des3 so I don’t have to enter passphrase every time sudo openssl req -new -key server.key -out server.csr sudo openssl x509 -req -days 365 -in server.csr -signkey server.key -out server.crt sudo cp server.crt /etc/ssl/certs/ sudo cp server.key /etc/ssl/private/

Now, place the following in the site configuration file (in /etc/sites-enabled/) before ==:

<pre class="src src-sh">SSLEngine on

SSLOptions +FakeBasicAuth +ExportCertData +StrictRequire SSLCertificateFile /etc/ssl/certs/server.crt SSLCertificateKeyFile /etc/ssl/private/server.key

For example, I changed my /etc/apache2/sites-available/www.mydomain.com from

<pre class="src src-sh"><span style="color: #ff4500;"># </span><span style="color: #ff4500;">Basic setup</span>

ServerAdmin my.email@my.domain.com ServerName www.mydomain.com DocumentRoot /home/user/www.mydomain.com/htdocs/

# HTML documents, with indexing.

Options +Includes

# CGI Handling ScriptAlias /cgi-bin/ /home/user/www.mydomain.com/cgi-bin/

Options +ExecCGI

# Logfiles ErrorLog /home/user/www.mydomain.com/logs/error.log CustomLog /home/user/www.mydomain.com/logs/access.log combined

to

<pre class="src src-sh"><span style="color: #ff4500;">## </span><span style="color: #ff4500;">following 2 for ssl</span>

NameVirtualHost *:443 NameVirtualHost *:80

# Basic setup ServerAdmin my.email@my.domain.com ServerName www.mydomain.com DocumentRoot /home/user/www.mydomain.com/htdocs/

# HTML documents, with indexing.

Options +Includes

# CGI Handling ScriptAlias /cgi-bin/ /home/user/www.mydomain.com/cgi-bin/

Options +ExecCGI

# Logfiles ErrorLog /home/user/www.mydomain.com/logs/error.log CustomLog /home/user/www.mydomain.com/logs/access.log combined

# Basic setup ServerAdmin my.email@my.domain.com ServerName www.mydomain.com DocumentRoot /home/user/www.mydomain.com/htdocs/

# HTML documents, with indexing.

Options +Includes

# CGI Handling ScriptAlias /cgi-bin/ /home/user/www.mydomain.com/cgi-bin/

Options +ExecCGI

# Logfiles ErrorLog /home/user/www.mydomain.com/logs/error.log CustomLog /home/user/www.mydomain.com/logs/access.log combined

# Authenticatiion

ScriptAlias /cgi-bin/ /home/user/www.mydomain.com/cgi-bin/

Options +ExecCGI

# Logfiles ErrorLog /home/user/www.mydomain.com/logs/error.log CustomLog /home/user/www.mydomain.com/logs/access.log combined

SSLEngine on SSLOptions +FakeBasicAuth +ExportCertData +StrictRequire SSLCertificateFile /etc/ssl/certs/server.crt SSLCertificateKeyFile /etc/ssl/private/server.key

Now I can use both http or https when accessing my site.

I run multiple sites on the same server. I wanted to use SSL on one or all of these sites, but that is not possible without having a static IP for each site. The reason is the HTTP header is encrypted, so Apache doesn’t know which site to take you to. See this and this for explanations.

Photo/picture/image repository, viewable using a web browser

I never had the habit of taking pictures and using them to cherish memories. I recently realized its importance when my mom passed away and I was scouring to find pictures. Luckily my siblings had some (I had to scan the the older, non-digital ones). These days, a camera is cheap, portable, and digital storage is cheap and only getting cheaper. I went out and bought a camera and will be taking pictures for the rest of my life now. I probably won’t print the photos all that much, just viewing, sharing, and storing them.

In terms of storage, I like to keep it simple and store them on my laptop using a directory hierarchy for albums. I currently view them with shotwell since it is the default for Ubuntu 10.10. It works quite nice actually. To share with my family (private), I upload the photos to Picasa Web. To share them with my social network, I upload them to facebook. If I want certain pictures to be public (viewable by the world), I upload them to flickr; these are usually taken by my phone.

Obviously the photos are scattered all over the internet with different photos chosen for each, with different quality, etc. It’s probably not easy to recover all the pictures as well. I shouldn’t have to as the main repository is my computer. However, I’d like this repository to also be viewable on the web to certain people (e.g., close family members). I’d like for them to be able to easily obtain the original pictures as well. I went out to look for a web server application for photo viewing and sharing, something analogous to WordPress for blogging, phpBB for forums, and Ampache for audios and videos. However, for photos, there isn’t an obvious application that stands out. My preferences for the software are:

  1. free, open source (GPL!),
  2. lightweight (as little dependency as possible and easy to setup),
  3. automatic resizing for thumbnails (I keep the photos in their original quality, and I don’t want to download massive files when viewing them on the web)
  4. privacy settings (I don’t want it to be viewable by anyone, only to a select few),
  5. local filesystem management of photos and albums via a directory hierarchy (just transfer/sync them into the designated folder and the website just works), and
  6. easy and batch downloading of photos and albums (if a sibling wants an album, just click a link to download an entire album).

I found some comparisons of applications here, here, and here. I asked on serverfault (originally on stack overflow but was moved over) but did not get much responses. I had to try a few out for myself.

Some things to note. A common theme to all the applications (besides a web server running, like Apache) are php (to generate website), MySQL (to store information like users, comments, etc.), and ImageMagick or GD (resizing images).

The privacy setting isn’t much of a concern since I can enable authentication in Apache adding an AuthType directive for the directory in the main config of the site (in /etc/apache2/sites-enabled/my-site) or in .htaccess of the restricted folder; the former is preferred to the latter if it is accessible for speed issues. I should also combine it with SSL just to be more secure (and to learn it).

Here are my evaluations (I just extracted their scripts into a folder of an existing website and followed the INSTALL instructions.).

  1. Gallery: seems to be the most popular, still actively developed, and has all the features I need (add downloading features via plugins) except for the local filesystem management of files (version 3.0). I talked to the main developer on irc and he told me it will probably be implemented if I requested it, and so I did. Let’s just say if the missing feature was there, this would be my chosen program (would’ve saved me a lot of time in testing other software too since this was the first software I tested).
  2. Piwigo: like Gallery, is also cutting edge and feature-rich. However, there are some things to note. You can manage pictures and albums from a directory, but you have to manually go into the website and sync (sync directories first, then dir + files); make sure to simulate the sync first, and change the privacy settings for the new images from the sync (e.g., admin only). For downloading, use the “Download Multi” plugin. Piwigo looks very nice, but the syncing issue is a pain and it took my a while to figure out how to set certain settings. I don’t think I will use it.
  3. Camera Life: also like Gallery and Piwigo. I didn’t try this out on my web server, but I know it does not have a privacy setting and does not have a batch downloading feature. I believe you can manage the images from the local system. I contacted the main developer Will, and he is very responsive. He plans to implement a privacy setting by the next release (March 2011) and will make the site friendly to batch downloading plugins on different web browsers.
  4. Bizou is dead simple and looks promising. Base on the demo, the only thing I don’t like is that when you click on a photo, it downloads the image in original quality. I would like the software to have an intermediate, web-browser quality version of an image to view (to save on bandwidth). It does not have a batch download mechanism, but I guess a web browser plugin would do.
  5. jGallery: did not try out since it is java based. I think files are managed by the local filesystem.
  6. linPHA: local filesystem management, but don’t think it has a privacy setting for viewing and batch downloading.
  7. Naig: looks simple, no batch downloading, no privacy setting, and local filesystem management. However, when I transferred a new folder into albums, I get an error when going to that album on the site. I might have settled for this initially if I didn’t get the error as it is very simple (good as a viewer of all pictures, include slideshow feature, and pictures are resized appropriately).
  8. phpGraphy: lightweight, no batch downloading feature, no privacy setting, but the filesystem management of pictures and albums is SOLID. This together with Authentication from Apache will probably be enough for a VIEWING repository. Basically, just move your images or albums into the pictures folder, chmod +rx the stuff, and the website will display everything properly.
  9. Zoph. Based on the website, I thought it had all the features I needed. The main thing to point out is Zoph offers some commands that you can use in the shell to import and organize (by albums, etc.). However, it is more complicated than I would want.

After evaluating the softwares above, I came to realize that I want an application that just watches a folder and displays them on the web. I want it to create thumbnails and web versions (appropriate resolutions) of the images automatically. The privacy setting can be set up in Apache so this isn’t a huge issue (although there is no logout button). Batch/mass downloading is not present in most of the software.

Here are my conclusions. I like Gallery best as it has all the features. However, just because I can’t manage the photos from a directory I will not be using it (yet). I really liked Naig due to its simplicity and how it displays the images; however, since it is currently broken for me, I won’t be using it. Hopefully the maintainer will get back to my inquiry.

I guess in the mean time, I can use phpGraphy with authentication from Apache. It is rock solid at what it does. However, no access to mass downloading or downloading files with the original quality. I can live without that for now.

Moved to WordPress for technical blogging

So I’ve decided to move my technical blog to wordpress, the .org version, not the .com version; that is, to host my own instance of wordpress. The move came from various reasons:

  1. syntax highlighting for soure codes,
  2. LaTeX support for my stat/math display,
  3. post via email,
  4. post via emacs,
  5. text-to-html syntax for blogging (think org-mode),
  6. easy to backup,
  7. all these features must be permanent (ie, the services above cannot be changed and my posts no longer look “right”), and finally,
  8. extensible

Points 4, 5, and 8 were never addressed with blogger, and point 2 was not consistent (contradicts point 7).

Posterous seemed very nice when i tried it out (especially point 3), but it didn’t seem too good for points 1 and 2 (although points 2 can be addressed by mathurl or Sitmo’s Equation Editor based on this post). I tried wordpress.com once, but thought it was inadequate. I knew it was possible to install your own wordpress version on a server but thought it was too much of a hassle. However, out of the blue I looked into the above features and found that wordpress had plugins (point 8) that really much fulfilled a lot of my needs.

I will outline how I installed wordpress on my school’s server and how I customized it to give me the features I want.

I pretty much followed the instructions from wordpress. My server already had apache on it, so I installed the following in addition:

<pre class="src src-sh">sudo apt-get update

sudo apt-get install php5 sudo apt-get install mysql-server sudo apt-get install php5-mysql ## mod_rewrite Apache module already installed wget http://wordpress.org/latest.tar.gz tar xvf latest.tar.gz mysql -u root -p ## enter password

In mysql, I have to set up the user and password:

<pre class="src src-sql">CREATE USER <span style="color: #ffa07a;">'ENTER MY USERNAME'</span>@<span style="color: #ffa07a;">'localhost'</span> IDENTIFIED BY <span style="color: #ffa07a;">'ENTER MY PASSWORD'</span>;

GRANT ALL PRIVILEGES ON . TO ‘ENTER MY USERNAME’@‘localhost’ WITH GRANT OPTION; EXIT

I also have to set up a database for the blog’s data by:

<pre class="src src-sh">mysql -u USERNAME -p

CREATE DATABASE wordpress;
GRANT ALL PRIVILEGES ON wordpress.* TO "ENTER MY USERNAME"@"localhost"
 IDENTIFIED BY "Enter WordPress Password";
FLUSH PRIVILEGES;
EXIT

We also need to update the wp-config.php file:

<pre class="src src-sh">emacs -q -nw wp-config.php <span style="color: #ff4500;">## </span><span style="color: #ff4500;">follow instructions in above site, generate random key, and set table_prefix to "snc_" for supernerdycool</span>

cd ~/ cd blog.nguyenvq.com cp -rf ~/Downloads/wordpress/* ./

sudo emacs -q -nw /etc/apache2/conf.d/virtual.conf sudo emacs -q -nw /etc/apache2/conf.d/virtual.conf ## make sure it looks like following: # # We’re running multiple virtual hosts. # NameVirtualHost * ## done

## make sure following is in /etc/apache2/httpd.conf: LoadModule rewrite_module /usr/lib/apache2/modules/mod_rewrite.so

sudo emacs -q -nw /etc/apache2/sites-available/blog.nguyenvq.com place following in it: # # Example.com (/etc/apache2/sites-available/adrc.nguyenvq.com) #

# Basic setup ServerAdmin EnterMyEmail ServerName blog.nguyenvq.com DocumentRoot /path/to/blog.nguyenvq.com

# HTML documents, with indexing.

Options Indexes FollowSymLinks MultiViews

## enable site sudo ln -s /etc/apache2/sites-available/blog.nguyenvq.com /etc/apache2/sites-enabled/blog.nguyenvq.com sudo /etc/init.d/apache2 restart

## go to http://blog.nguyenvq.com/wp-admin/install.php in web browser

I initially had some errors with mysql and php, but this showed me that I was missing mysql-php in the install (apt-get); restart apache again.

To install plugins, it is easiest to have an ftp server set up for the server. I installed it via “apt-get install proftpd ## select standalone.”

I addressed what I wanted above with:

  1. SyntaxHighlighter Evolved,
  2. wp-latex,
  3. postie (require php5-imap for imap) or wordpress’s native email support (this didn’t work too well for me’) – email does not work well with wp-latex and syntaxhighlighter as this modifies the incoming text,
  4. weblogger mode in emacs (turn on xml-rpc in wordpress),
  5. Markdown for WordPress and bbPress
  6. backup by using rsync (for wordpress files) and mysqldump for the database (data),
  7. since these are installed on my own server, of course all these services are permanent (of course I can move them to another server too),
  8. other plugins: Akismet, WP-reCAPTCHA, Acronyms, Search and Replace (very good for updating all posts), Search Unleashed(uncheck “Some themes don’t display any search result content. Enable this option to force the theme to display results” in options or when visiting post from google search result, syntax highlighting will get screwed up), ShareThis (share to twitter, facebook, …, all social media), and Simple Image Widget.

The SyntaxHighlighter Evolved and Markdown didn’t play nicely together. This post’s comments has the solution, by wrapping the source code chunk with the div tag and changing the add_filter line in markdown to 9 (see below). Since this is hard to display, I will describe the characters in text:

<pre class="src src-sh">LeftAngleBracket div RightAngleBracketDiv LeftSquareBracket bash RightSquareBracket

my code LeftSquareBracket FORWARD SLASH bash RightSquareBracket LeftAngleBracket FORWARD SLASH div RightAngleBracket

Also change markdown.php:

<pre class="src src-php">add_filter('the_content_rss', 'Markdown', 6);

to

add_filter(‘the_content_rss’, ‘Markdown’, 9);

syntax highlighter’s is less than 9

See this site for the lnaguages supported.

For LaTeX, include code by:

<pre class="src src-sh">DOLLARSIGN latex ENTER LATEX CODE DOLLARSIGN

to get \(latex y=x’\beta_0\).

For Markdown syntax, look at this site for a quick reference. The main thing is to use two asterisks (\/\/) around the text for bold and one asterisk (\*) around the text for italics. For links, use

<pre class="src src-sh">LeftSquareBracket TEXT RightSquareBracket LeftParen URL RightParen

. There are a lot more, but these will be commonly used. I would like the syntax to be like that of org-mode, but I can’t have everything. I could hack the php file to modify the syntax like org-mode’s, but I’m too lazy now and don’t want to deal with the regexp’s.

I also like the notepad theme as it fits my technical blog quite nice.

backup wordpress site:

<pre class="src src-sh">rsync -av --modify-window=1 --delete username@myserver:location localLocation

## this backs up to my computer

backup database and some cron scripts (multiple scripts in the following):

<pre class="src src-sh"><span style="color: #ff4500;">### </span><span style="color: #ff4500;">http://codex.wordpress.org/Backing_Up_Your_Database</span>

mysqldump –add-drop-table -u MyUsername -p Databasename | bzip2 -c > blog.nguyenvq.com.bak.sql.bz2 ## wordpress is the database name, -p says to specify password after ## now enter password

#### restore database ### http://codex.wordpress.org/Restoring_Your_Database_From_Backup bzip2 -d blog.nguyenvq.com.bak.sql.bz2 mysql -u MyUserName -p Databasename < blog.nguyenvq.com.bak.sql ## enter password

#### setup cron job on server Backup_blog.nguyenvq.com_DB.sh: #! /usr/bin/env bash mysqldump –add-drop-table -u MyUsername –password=MYPASSWORD Databasename | bzip2 -c > /path/to/blog.nguyenvq.com.bak.sql.bz2

chmod +x Backup_blog.nguyenvq.com_DB.sh chmod o-r Backup_blog.nguyenvq.com_DB.sh

crontab -e: 00 12 * * * /path/to/Backup_blog.nguyenvq.com_DB.sh 59 23 * * * /path/to/Backup_blog.nguyenvq.com_DB.sh ## need a carriage return at end of file

#### setup cron job on my computer ## make sure passwordless ssh is on Backup_blog.nguyenvq.com.sh: #! /usr/bin/env bash INCREMENT=date +%Y%m%d%H%M DIR=“$HOME/path/to/wherever” DIR=“$HOME/path/to/wherever/Wordpress_blog.nguyenvq.com” # -q for quiet in cron rsync -q -av –modify-window=1 –delete username@server:/path/to/blog $DIR0/ scp -q username@server:/path/to/nguyenvq.com.bak.sql.bz2 $DIR/blog.nguyenvq.com.bak.sql.$INCREMENT.bz2 find $DIR -mtime +14 -exec rm -f {} \; ## delete files older than 14 days

crontab -e: 02 12 * * * /path/to/Backup_blog.nguyenvq.com.sh 02 00 * * * /path/to/Backup_blog.nguyenvq.com.sh