Running Margarita in Apache

Updated Nov 18, 2013 – updated for Ubuntu 13.10 and Apache 2.4; including suggestions from Brandon Kerns, submitted in the comments. Thanks Brandon!

Lots of web apps are starting to switch from PHP to Python for the backend, and with good reason, but one thing that’s always bothered me is how many people don’t run their Python apps in Apache. Most people find it easier to run these web apps using a development-grade server such as the stand-alone WSGI server commonly used in Django or Flask projects. Generally, this comes with the follow-up task of making sure the web app’s WSGI instance will automatically launch on boot. Then of course there’s the fact that these server stacks were designed to make development easy; they were never meant to run in production. For that, there’s mod_wsgi.

In this post, I’m going to walk through the process of installing Reposado with Margarita as the front-end on an Ubuntu server host. I’m just going to jot down notes as I walk through the process on a fresh Ubuntu Server 13.10 installation, so ymmv, but not by much. I won’t go through the process of installing Ubuntu, but just know that the only option I enabled during installation was the SSH server. The rest, follows:

Installing Required Software

As a matter of good practice, we’re going to make sure our Ubuntu server is fully patched before we start. Then we’ll install mod_wsgi as well as the git command line tools and python’s setuptools package. Since Margarita runs on Flask, we’ll need to install that as well. Apache2 will be installed as a dependency of mod_wsgi, so I’ve omitted it from the following commands.

sudo apt-get update
sudo apt-get upgrade
sudo apt-get install libapache2-mod-wsgi git python-setuptools
sudo easy_install flask

You can install Reposado and Margarita anywhere you’d like, but I’m going to use /usr/local/ASUS (which stands for Apple Software Update Server) just to keep things organized. The following commands will create the reposado, margarita, www and meta directories within /usr/local/ASUS. The www directory will be the location from which reposado’s catalogs and downloads will be served, and you can think of the meta directory as reposado’s work area.

Clone the code and setup the directories:

sudo mkdir /usr/local/ASUS
cd /usr/local/ASUS
sudo chown  .
git clone https://github.com/wdas/reposado.git
git clone https://github.com/jessepeterson/margarita.git
mkdir www meta

You’ll notice that I had you chown the directory so that you own it. This isn’t required, but it eliminates a bunch of extra ‘sudo’ calls for the rest of the steps.

Next we’ll need to configure Reposado and let it sync, and I’m going to do so without replication. If you want replication so your clients will download updates from your server instead of Apple’s, you’ll need to enter your host’s FQDN for the answer to the last prompt, e.g. http://su.example.com

Configure Reposado:

./reposado/code/repoutil --configure
Filesystem path to store replicated catalogs and updates [None]: <strong>/usr/local/ASUS/www</strong>
Filesystem path to store Reposado metadata [None]: <strong>/usr/local/ASUS/meta</strong>
Base URL for your local Software Update Service
(Example: http://su.your.org -- leave empty if you are not replicating updates) [None]:
./reposado/code/repo_sync

Although we haven’t setup Apache to actually serve anything yet, you’ve got Reposado fully installed and configured so, as Greg would say, “Good for you!”. Now we can move on to setting up your Margarita front-end. We’ll start things off by borrowing from Jesse’s instructions, just to make sure things have been properly installed. Since Margarita and Reposado are both written in Python and share common tasks, it only makes sense that code is reused where possible; that’s exactly what Jesse has done. So in order for Margarita to use Reposado’s code, it needs to be able to find it. We’ll need to create a few symbolic links to do this.

Let Margarita access Reposado’s shared resources:

ln -s /usr/local/ASUS/reposado/code/reposadolib margarita/reposadolib
ln -s /usr/local/ASUS/reposado/code/preferences.plist margarita/preferences.plist

At this point Margarita should be completely installed and configured. To test, run the following command and then point your favorite browser to http://example.com:8089 (don’t worry, port 8089 is just for this test). If all goes well, Margarita should load but without showing any updates. To see the updates, uncheck the “Hide commonly listed updates” button at the top of the page. If you still don’t see any updates, you’ve encountered a problem and should look at the output in your terminal window to start troubleshooting.

Testing Margarita:

python margarita/margarita.py

Running Margarita in Apache

So far we’ve properly configured both Reposado and Margarita. Now all we want to do is make sure the web interface will automatically come back to life when the server is rebooted. We could write a custom service that uses Python to launch the margarita.py script as we’ve done in the above test, but we already have Apache running to serve the software updates, so why not use that to serve the Margarita web interface as well?

Before we get started on this section, I should mention that these instructions assume you have no other use for your Apache instance on your server outside of serving software updates. If you do, you should look into using a different VirtualHost directive instead of performing a complete takeover of Apache as we’re about to do.

Creating Our Very Own .wsgi Script

A .wsgi script gives mod_wsgi the information it needs to launch the python web app, but Margarita doesn’t come with one. Fortunately, these files are pretty easy to make. Using your favorite text editor (*cough* Vim *cough*), create the file /usr/local/ASUS/margarita/margarita.wsgi with the following contents:

/usr/local/ASUS/margarita/margarita.wsgi

import sys
EXTRA_DIR = "/usr/local/ASUS/margarita"
if EXTRA_DIR not in sys.path:
    sys.path.append(EXTRA_DIR)

from margarita import app as application

Configuring Apache

Before we go about configuring Apache, we need to make sure it has the proper filesystem permissions.

sudo chgrp -R www-data /usr/local/ASUS
sudo chmod -R g+r /usr/local/ASUS

The following snippet represents the entirety of my /etc/apache2/sites-enabled/000-default.conf file. If you copy and paste, you should see that everything is working properly after you restart apache, generally with sudo apachectl restart. However, this setup will allow anyone to wander by and configure your ASUS server.

So, before you copy and paste the snippet below: You should look into securing your web server via SSL as well as setting up some basic authentication mechanisms.

/etc/apache2/sites-enabled/000-default.conf

<VirtualHost *:80>
    ServerAdmin webmaster@localhost
    DocumentRoot /usr/local/ASUS/www

    # Base cofiguration
    <Directory />
        Options FollowSymLinks
        AllowOverride None
    </Directory>

    # Reposado
    Alias /content /usr/local/ASUS/www/content
    <Directory /usr/local/ASUS/www/>
        Options Indexes FollowSymLinks MultiViews
        AllowOverride None
        Require all granted
    </Directory>

    # Margarita
    Alias /static /usr/local/ASUS/margarita/static
    WSGIDaemonProcess margarita home=/usr/local/ASUS/margarita user=www-data group=www-data threads=5
    WSGIScriptAlias / /usr/local/ASUS/margarita/margarita.wsgi
    <Directory /usr/local/ASUS/margarita/>
        WSGIProcessGroup margarita
        WSGIApplicationGroup %{GLOBAL}
        Require all granted
    </Directory>

    # Logging
    ErrorLog ${APACHE_LOG_DIR}/asus-error.log
    LogLevel warn
    CustomLog ${APACHE_LOG_DIR}/asus-access.log combined
</VirtualHost>

Done.

Well, that’s the plan anyway. If you’re still having trouble getting things working, here are a few resources to get you started down the troubleshooting path:

Addendum 1: Scheduling repo_sync

Out of the box, reposado will not run the repo_sync command without your direct invocation. If you want your new SUS server to look for any new updates released by Apple on its own, leaving you to simply approve them, you can setup a simple cron job. Since it’s probably sane for most environments to simply run this script once per day, fire up a sudo vim session and…
sudo vim /etc/cron.daily/repo_sync

#!/bin/bash
pushd /usr/local/ASUS/reposado/code/
./repo_sync
/bin/chgrp -R www-data /usr/local/ASUS/www
/bin/chmod -R g+rX /usr/local/ASUS/www

…and of course, make sure the script is executable with sudo chmod +x /etc/cron.daily/repo_sync.

Addendum 2: Keeping Up To Date

Every once in a while, Apple will throw a curveball at Reposado which requires a code modification. When that happens, you can easily upgrade both Reposado and Margarita via the git command.

cd /usr/local/ASUS/reposado
git pull
cd /usr/local/ASUS/margarita
git pull
sudo apachectl restart
Advertisements

14 comments

  1. Thanks for the great tutorial. I’m wondering if you have tried the basic security you have links too? I tried the .htaccess method and was unsuccessful. Margarita appears to load out ahead of the www files. Thanks for any assistance you can offer.

    1. I have. I’m currently using .htaccess to prompt for authentication in my environment. I’m not sure what you mean by “Margarita appears to load out ahead of the www files”, so the only comment I’ll make on that is this: Apache is serving Margarita, so there’s no way for Margarita to load before Apache or its configs. If you could expand on that comment, perhaps I can be of further assistance.

  2. I may just be uncertain where to add the .htaccess file. I created it in the /ASUS/www folder. I also tried the other method mentioned in the link. The security window fails to load and when I move .htaccess up to the /ASUS level a 500 error occurs. I built the server twice in AWS and on ESXI at home locally with the same results. Thanks for responding.

  3. I followed this to the T (I hope!) but I can only get it partially working. Margarita works perfect, and I can create branches, sync, etc. But I can’t view a catalog in the web browser. I’m using http://mysever.com/content/catalogs/others/index-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog but nothing shows up. I have tried every catalog in there. And I started form scratch 3x just in case I made a silly mistake somewhere. Any ideas?

  4. I played around with this some more. I commented out the section on Margarita in the apache config file and I was then able to view the sucatalog files in a web browser. I can also configure my clients to use the su server. However, no dice on Margarita. As soon as I uncomment that section Reposado breaks.

  5. Maybe. In the error.log (not asus-error.log) there is this mod_wsgi error:

    [Mon Sep 23 17:24:35 2013] [warn] mod_wsgi: Compiled for Python/2.7.3.
    [Mon Sep 23 17:24:35 2013] [warn] mod_wsgi: Runtime using Python/2.7.4.
    [Mon Sep 23 17:24:35 2013] [notice] Apache/2.2.22 (Ubuntu) mod_wsgi/3.4 Python/2.7.4 configured — resuming normal operations

    Otherwise I get basically the same line in asus-access.log:

    When I comment out Margarita:
    192.168.40.162 – – [23/Sep/2013:17:28:34 -0400] “GET /content/catalogs/others/index-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog HTTP/1.1” 206 748624 “-” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.47 Safari/537.36”

    When I enable Margarita:
    192.168.40.162 – – [23/Sep/2013:17:16:27 -0400] “GET /content/catalogs/others/index-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog HTTP/1.1” 404 442 “-” “Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/30.0.1599.47 Safari/537.36”

    Essentially I get 404 errors whenever I try to access anything in www folder for Reposado with Margarita enabled, but I can access everything when Marga is disabled.

    I guess I can run it in flask on port 8089.

    Any help would be amazing though.

  6. Not sure if my previous “guest” comment ever showed up, but I was having the exact same issue that Kyle is describing.

    I was able to get everything to work by changing the the WSGIScriptAlias from “/” to “/margarita”. It seems that when wsgi is directed to the root, it takes it over completely. Only oddity now is that in the URL I have to go to http://1.2.3.4/margarita/ *with the trailing slash* to get margarita to come up. If I go to the same URL without the trailing slash, margarita just hangs with the loading bar. Hope it helps!

  7. Thanks Brandon! That totally works.

    What I eneded up doing is breaking up reposado and margarita into two separate sites on the same server. Then I didn’t need to use the trailing slash or /margarita at all because now my sites were reposado.myserver.com and margarita.myserver.com.

  8. Really glad I found this Post. I followed it as well and ran into an issue I see regarding Apache2. The dependencies don’t all load apache2s programs, for example I get:

    /etc$ apache2ctl -V
    The program ‘apache2ctl’ is currently not installed. You can install it by typing:
    sudo apt-get install apache2.2-common

    Going to try fresh again with apache2 install at the first apt-get

  9. Hi,

    Great guide – thanks! Just a minor correction, it seems that the first part of the Directory-tags has been cut short in the apache-site-configuration file:

    ——– SNIPPET ——-
    # Reposado
    Alias /content /usr/local/ASUS/www/content
    usr/local/ASUS/www/> <— HERE
    Options Indexes FollowSymLinks MultiViews
    AllowOverride None
    Require all granted

    # Margarita
    Alias /static /usr/local/ASUS/margarita/static
    WSGIDaemonProcess margarita home=/usr/local/ASUS/margarita user=www-data group=www-data threads=5
    WSGIScriptAlias / /usr/local/ASUS/margarita/margarita.wsgi
    usr/local/ASUS/margarita/> <— HERE
    WSGIProcessGroup margarita
    WSGIApplicationGroup %{GLOBAL}
    Require all granted

    ——- /SNIPPET ——-

    /Steffan

    1. I haven’t tested (a.k.a. ‘proofread’) the update, but I believe it fixes what you’re reporting. Let me know if not. Thanks!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s