February 2010

It is a good discipline to read machine logfiles periodically and be on the lookout for messages that stand out.

One recent linux message that might start appearing on a Linux 64 bit machine concerns HPET ( High Precision Event Timer ) and here is an example:

CE: hpet increasing min_delta_ns to 15000 nsec

Now running off and web searching for results is probably a longer way round than reading the next paragraph or two, which have the important things you might want to know first.

Firstly this is a Linux kernel feature designed to help power efficiency so on the whole it is a positive thing.

It is neither brand new nor some serious bug.

It can be disabled if you are bored of reading about it already ( hpet=disable )

There is an Intel presentation titled ‘Getting maximum mileage out of tickless’ which has on page 3 a gentle introduction to why hpet is a desirable feature.

For the more technically adept there are some further details here and here.

Back to the message I was seeing in my logs…there are bug reports regarding messages of the sort 'hpet increasing min_delta_ns', in particular, if you are seeing lots of these messages in your logs, then it might be worthwhile engaging with the kernel development team and sharing your output.

For me this laptop install is a brand new Xubuntu 64 bit install and is still in the process of being fully configured. When I have all my applications running as I wish then I’ll check the logs again and see if things look okay. With just a single message appearing occasionally, it might be that waiting a few months for the Lucid upgrade is (and the newer kernel 2.6.32) will be the path of least effort.

For folks running Debian / Ubuntu servers in the cloud, this information regarding tickless might be worth a read.

An example of a Python Server Page:

<html><body><p>Python Version is:
<% import sys %>
<%= sys.version[:3] %>

( put the above code in a file named version.psp and you are on your way )
Here is another:

<html><body><p>Hello you at IP: <%= req.connection.remote_ip %>

( put that code in a file named hello_you_at.psp and you are on your way )

What is a Python Server Page?
It is a mix of raw html and python generated output.

Which Apache Handler do I need to use for my Python Server Pages to work?

( There are several tutorials on how to enable the Handler mod_python.psp and I do not wish to cloud this article in repeating that )
Hint: PythonHandler mod_python.psp | .psp or simply PythonHandler mod_python.psp is a big part of the story

But I read somewhere about the Handler named publisher?
That is not for Python Server Pages – when using that handler you would not be mixing raw html and <% %> enclosed python.

But if I want to be able to process arguments like ?nextpage=logout then I need publisher handler don’t I?
Not at all. Python Server Pages have the form object and can check if form.has_key('nextpage') and suchlike – see example at end

What are the possible reasons why Python Server Pages are sometimes misunderstood?

  • Perhaps the people who are writing the documentation you read might be more familiar with ‘modern’ frameworks that are thought more swish than Python Server Pages, or considered more contemporary (Django, Gears this n that, whatever)
  • The fact that the Apache module mod_python gives several Handlers that can be used ( mod_python.psp, mod_python.publisher, and more. ), might lead to a little confusion.
  • Maybe the people who are writing the documentation you have read are understanding CGI well, but are a bit light on the subject of Server Page technology in the style of PSP, PHP, ASP

Taking a quick look at “How to use python on the web” shows (today) little mention of Python Server Pages, and seems almost to encourage PHP folks to abandon the “model” of raw html mixed with <% %> embedded python.

Before writing the next couple of points I think it feels correct to say that thanks are due to Marek Kubica for documenting “How to use python on the web”. Sometimes documentation is seen as the less sexy part of contributing to a community, and the job of the being the draftsman puts you in line for (hopefully constructive) criticism on what you might have missed out/corrections.

The mod_python section of the documentation – a couple of points:

Their first thought is mostly mod_python because they think that this is the equivalent to mod_php. Actually it is not really.

I have difficulty agreeing totally with the above. Python Server Pages can be a pretty close match to the ‘raw html and embedded style’ a JSP, or PHP programmer might be used to.

As a developer who has worked with Java Server Pages in the past, I have no problem in understanding that an interim file is generated from the .psp file. It seems that the interim file is refreshed when the .psp file timestamp is modified so no worries there about cache items being stale.

These are the reasons why mod_python should be avoided when writing new programs.

I fail to see those reasons and they seem not to be explained. If the argument might be “…application logic inside the presentation component…” then that argues against JSP and PHP style development just as it argues against Python Server Pages.
( It seems a little harsh on the developers of mod_python.psp to make such a statement at all )

The current documentation “How to use python on the web” is a reasonable overview of the different ways of using Python in your webserver, however I just feel that a separate section for “mod_python Python Server Pages” and “mod_python publisher” might help newcomers find things more appropriately.

Just for the record, I personally do not have knowledge of the different frameworks that are required to produce such a comprehensive document as has already been produced (mentioned above), but in experimenting with Python Server Pages, I perhaps do have an outlook that WSGI, or other CGI favouring people might not have.

Eventually I will learn more about the CGI technologies and the options there, but for now mod_python version 3.3 and Python Server Pages is proving a great way to experiment with Python and Apache.

If you are still interested in Python Server Pages and want a more advanced example then here is an example that uses an optional argument nextpage:

<%= ‘Hello. My nextpage will be: ‘ %> <% nextpage_string = ‘cannot say!’ if form.has_key(‘nextpage’): # consult argument nextpage_string = form[‘nextpage’].lower() #nextpage_string = ‘can say can say’ # end of indent indicator is essential %> <%= ‘…’ %> <%= nextpage_string %>

( put the above code in a file named hello_nextpage.psp and try calling it as nextpage.psp?nextpage=logout )

If you are intending to get a bit more advanced with Python Server Pages then you should be aware that, because Python itself is particular about indentation, then you will have to take extra care in your PSP pages. If you have not worked with Python much before then that might well bother you.

If you are planning to use for loop and advanced constructs, then you will maybe want to join the mod_python mailing list or at least have had a browse through an article about indentation, or some of the postings about indentation.

Apache and mod_python:

Know in advance which of the MPMs you are going to use – prefork, worker, or otherwise.

If you know your application is threadsafe then worker might be the MPM for you.

However if you are using a web application framework, then it is the web application framework that must be threadsafe, and the notes page for the framework should give you an indication.

Many web application frameworks are not guaranteed thread-safe, and as a result recommend the prefork MPM.

Books on ‘Concurrent programming’ are available, but be warned ‘ This is a serious book about a complex subject

Web server performance can be better with the worker MPM, but only if the developers of the framework have taken the trouble to write quality code that has considered thread-safety. Most web application frameworks started out with little consideration for thread-safety, and are forced to recommend the least efficient MPM as a result.


Notes and Further reading:

Here is a 2011 quote from Wikipedia page for mod_python:

“This does not mean that it is ‘dead’ as some people have claimed. It simply means that the code and the project are mature enough when very little is required to maintain it.”

Further Reading:

Personal thoughts on mod_python

  • Use it if you want
  • Expect proponents of WSGI to try to convince you to learn WSGI (hey it is an additional skill so why not)
  • As with all web technologies and frameworks it has some drawbacks
  • Restarting an Apache server to read developer code updates is an inconvenience – probably makes it unsuitable for a shared hosting situation

2012 update for users of the Django web framework:

Django 1.4 was released March 2012. Planning for the version after 1.4 is underway, and I include some notes from the Django site:

Support for mod_python has been deprecated, and will be removed in Django 1.5

For Current Django (1.4) and earlier:

The mod_python module for Apache can be used to deploy Django to a production server, although it has been mostly superseded by [what we feel for Django] is a simpler mod_wsgi deployment option.

If you are using a particular web framework, and want to continue using it, then you need to have awareness of the system requirements for that framework. Those quoted extracts are a part of that.

Version 1.0 of Django and later, have the goal of being thread safe. The later the version the better, so if you are using Django 1.2 or newer, then there should really be no real need to run under Apache ‘prefork’ rather than using ‘worker’.