Pingly – a Roxen module for automatic Twingly pinging
Twingly is a blog search engine that focus on indexing the “blogosphere” rather than being a generic search engine. Twingly has a ping service that let you ping Twingly when you have new content on your blog so that Twingly can head over there and index the new content asap.
Since Roxen CMS have event hooks this module listens for newly published files and when found automatically notifies Twingly about it.
The only thing needed is to set up a config file in the SiteBuilder’s workarea so that this module knows under which paths newly published content should notify Twingly and with which arguments. But all this is documented in the module.
One note! If you run a replicated environment install this module on one of the frontend servers, not the backend. If installed on a backend Twingly might head over to your site before the new content has been replicated.
Gravatar module for Roxen
Although I’m working on some social web related Pike modules where a Gravatar module is included I needed a Gravatar module right now at work so I hacked up a standalone Gravatar Roxen module.
This is how it works
- <ul>
- <emit source="sql" host="mydb"
- query="SELECT name, email, `date`, body FROM my_comments"
- >
- <li>
- <gravatar-img email="&_.email;" rating="pg" size="32"
- default-image="/path/to/icon.png"
- /> &_.name; wrote at <date iso-time="&_.date;" />
- <wash-html paragraphify="">&_.body;</wash-html>
- </li>
- </emit>
- </ul>
Download the Gravatar Roxen module 14:20, Wed 09 December 2009 :: 7 kB
UTF-8 encoding/decoding in C
I was working on a simple database to Excel XML exporter the other day and decided to write it in C. Now, the problem was that since the Swedish language contains non-ascii characters the output needs to be UTF-8 encoded. C doesn’t have a built-in function for this – it seems I should add since I’m a C rookie – and no matter how I searched at Google I couldn’t find anything useful. So I thought…
Look at PHP
…why not look at the source code of PHP and see how the PHP functions utf8_encode and utf8_decode are being done. So I downloaded the source of PHP and with a little find . -name *.c -print | xargs grep "utf8_encode" I found the functions in xml.c. Thankfully they weren’t too complicated – when dug out from the rest of the XML functions – so I didn’t take too long before I had them as standalone functions.
This is how they are used:
- #include "utf8.h"
- int main(int argc, char **argv)
- {
- char *iso_str = "Pontus Östlund";
- char *utf8_str;
- utf8_str = utf8_encode(iso_str);
- iso_str = utf8_decode(utf8_str);
- return 0;
- }
And it seems to be working quite OK!
utf8.c 17:31, Sat 17 October 2009 :: 5.3 kB
utf8.h 17:31, Sat 17 October 2009 :: 1 kB
Firefox Domain-Swapper extension
Anybody working on a replicated web site (with an edit/backend server and one or more frontend servers) with perhaps lots of sub sites has probably experienced the hassle of switching domains between the backends and frontends.
Say you get an email saying: Can you change … to … on http://some-front-end.com/some/path/to/a/page.html. You click on the link, since you’r lazy, and then have to change the domain in the address bar to your backend domain and 40% of the time you misspell and have to go up there again and…
Anyhow! I got tired of this so I created a domain swapping extension to Firefox. It adds a context menu (right-click on any page) where you can choose to change to another domain – the URL will stay intact.
So if you are working with replicated sites the extension is free to download at:
http://www.tekniskaverken.se/fishnet…pper/index.xml
A couple of wallpapers
Lately I have found a new time waste – playing with Photoshop and Gimp to create desktop wallpapers. I must say Gimp isn’t too bad although it’s quite far behind Photoshop when it comes to usability and especially layer styles. I downloaded a plug-in that works pretty well and similar to the effects in Photoshop, but the big difference is that the effects will be put in separate layers in Gimp which means you can’t fine tune them in real time afterwards. And one really misses the layer folders! Anyway, Gimp works pretty well and when you get used to the differences from Photoshop and you learn the shortcuts it’s quite usable after all.
All wallpaper are in 1680×1050px
Chocolate – Made with Photoshop

Fairytale – Made with Photoshop (image and butterfly from Stock exchange)

Gnome Shield – Made with Photoshop

Roxen Application Launcher 0.3
I just created an updated version of Roxen Launcher. I got an error report from one who tried to build the application:
./ApplCon.cs(232,24): error CS0122:
‘System.Net.Sockets.TcpListener.Server’ is inaccessible due
to its protection level
An explanation of the error can be found at MSDN. The reason seems to be due to TcpListener.Server being a protected property but I tried to access it as public. The strange thing is that I didn’t get an error about it nor on my machine at home or at work. Now I created a derived class of TcpListener.Server so that I can access the protected property.
We’ll see if it works!
Roxen Application Launcher 17:31, Sat 17 October 2009 :: 97.3 kB
Roxen Application Launcher for Linux
I thought I should broaden my C# knowledge a bit and you know how it is: To learn new stuff you need a real project to work on or else you will lose the fire sooner than later. So I came up with a good project that is actually useful to me: Porting Roxen’s “Application Launcher” to C#. There’s nothing wrong with the original one, written in Pike, except that it uses GTK 1 which is quite hideous (in an aesthetic meaning) compared to the newer GTK 2. And I also though it would be cool to create a panel applet (in the notification area of Gnome so you could put the Application Launcher in the background).
BTW: For those of you not knowing what the heck Roxen’s Application Launcher (AL here after) is here’s a brief explanation: Files in Roxen CMS is stored in a CVS file system which means that you don’t deal with files the way you normally do. To manage files you use a web browser interface (which is a darn good one I might add) but sometimes you actually want to edit files in your standard desktop application. And it is here the AL comes to play. You can download a file through the browser interface so that the file is opened in the AL. AL will then open the file in the desktop application you have associated with the file’s content-type. When you make your changes and saves them the AL will directly upload the changes to the server. So in short I could have said: The Application Launcher is a means to edit files on a remote Roxen server with a preferred desktop application.
The obstacles
I must say I’ve learned a lot from this project!
First off: If you download a file for editing and the AL is already started you don’t want to start a new instance of AL (this is something I have never ever thought about before – in general terms, not just concerning AL) but when you do think about it you find that it’s not a piece of cake to solve. I solved it the way it is solved in the original AL. The first instance of AL that is started also starts a “socket server” that listens for incoming traffic on a given port on the local IP. When a new instance is started it first checks if it can connect to said port and if it can it sends the arguments through the socket to the first instance which then handles the request. The second instance is simply terminated when it has send the data though the socket.
So there I had to do some socket programming. Great fun
Secondly: Stuff happens in the background of AL – data send through the socket remember – which means that nothing happens when you try to update the Graphical User Interface. (NOTE! This is the first more advanced desktop application I’ve done.) After “Google-ing” around a bit I came to know that this was a real newbie problem
The thing is that the GUI can only be updated through the same thread that started it so when using background threads – implicitly that’s what I’m doing although handled by the asynchronous callback infrastructure of C# – you need to make sure the GUI is updated through the main thread. This is the most simple way so solve it:
- Gtk.Application.Invoke(delegate{
- CallFunctionToUpdateGUI();
- });
That’s not too difficult when you know it
Thirdly: The AL is sending data back and forth through the HTTP protocol which means we have to use some sort of HTTP client. C# has a couple of ways to do this but unfortunately they came up short, or I couldn’t use them anyway. I didn’t manage to figure out exactly why I always caught an exception saying something like: A protocol violation occurred!. I’m far from the only one who have fought with this and it has something to do with the headers sent from the remote server. You can invoke “unsafe header parsing” but that was to much of a hassle so I created my own little HTTP client.
One big annoying thing with C# is that is seems almost impossible to turn data from streams into strings without having to use any one of the System.Text.Encoding.* classes/objects which in my case meant that images and files in binary form got seriously fucked up. I manged to solve this my never turning the data into a string but keeping it as a System.Text.Encoding.* all the way from request to response to saving to disk. It was rather irritating but at the same time nice when solved (and I learned a whole bunch about System.Text.Encoding.*, System.Text.Encoding.*, System.Text.Encoding.* and System.Text.Encoding.*.)
Finally: Of course I learned a great deal more about C# but this blog post is starting to get pretty excessive so I will round it off by saying that MonoDevelop is starting to become pretty darn good! I just upgraded to the latest version of Ubuntu and that also meant that I got the latest MonoDevelop and I must say it’s more stable than ever (although it occasionally crashes) and a whole bunch of new features are in place. One I havn’t used before – although it might have existed before – is the “Deployment” stuff. It creates a package with configure and make files for optimal compilation. Really smooth!
Source and screens
I will finish off by adding the source files and a few screen shots:
Roxen Application Launcher 17:31, Sat 17 October 2009 :: 96.7 kB
Screen shot 1: Just a standard view

Screen shot 2: The panel applet in action

Script for adding all new files in a SVN repository
When working with revision controlled stuff you sometimes add loads of new files to an already existing repository. Then when you are about to check in the next revision you have forgotten where all the new files are and you probably don’t feel like writing svn add thefile too many times. This is of course no problem if you’r using a decent SVN client – like Tortoise SVN on Windows – but I prefer the command line so the solution was writing a script that adds all new files in a directory structure to the SVN repository.
There are “one liners” for this but I wanted a bit more flexibility and ease of use so I wrote the a script that also lets you define regexp patterns for files you don’t want to add even if they exist in the directory structure.
This is how to use it:
- # Add all new files
- svnadd path/to/repository/
- # Add all new files except those matching the regexp
- svnadd -s ".*.txt|tmp/.*" path/to/repository/
- # Add all new files except those matching the regexp in
- # the file defined by the "-f" flag
- svnadd -f regexp.txt path/to/repository/
Quite easy! If you don’t feel like remembering the pattern for files to skip, put the regexp in a file and use the svn add thefile flag.
- #!/bin/bash
- ##########################################################################
- #
- # Script for adding all new files in a directory structure to an SVN
- # repository.
- #
- # copyright © 2007 Pontus Östlund <spam@poppa.se>
- #
- # The svnadd script is free software; you can redistribute it and/or
- # modify it under the terms of the GNU General Public License as published by
- # the Free Software Foundation; either version 2 of the License, or (at your
- # option) any later version.
- #
- # The svnadd script is distributed in the hope that it will be useful,
- # but WITHOUT ANY WARRANTY; without even the implied warranty of
- # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General
- # Public License for more details.
- #
- # You should have received a copy of the GNU General Public License
- # along with this program; if not, write to the Free Software Foundation,
- # Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
- #
- ##########################################################################
- while getopts "s:f:h" opts
- do
- case $opts in
- s) skip="$OPTARG";;
- f) if [ -f "$OPTARG" ]; then
- skip=$(head -n 1 $OPTARG)
- else
- echo "$OPTARG doesn't exist!"
- fi
- ;;
- h) help=1;;
- esac
- done
- shift $(($OPTIND - 1))
- if [ -z $1 ] || [ "$help" ]; then
- echo "About: svnadd enables you to add all new files in [dir] to a "
- "subversion repository"
- echo "Usage: svnadd [flags] [dir]"
- echo
- echo " -s <pattern> Regexp pattern for files to skip"
- echo " -f <path> Path to file containing regexp for files to skip"
- echo " The regexp should be placed in line one!"
- echo " [dir] The path to the repository"
- echo
- echo "Example: svnadd svnsrc/"
- echo "Example: svnadd -s 'tmp/.*.xml' svnsrc/"
- echo "Example: svnadd -f regexp.txt svnsrc/"
- echo
- exit 0
- fi
- dir=$1
- if [ ! -d "$dir" ]; then
- echo "$dir is not a directory"
- exit 1
- fi
- while read status file; do
- if [ "$status" = "?" ]; then
- res=$( echo "$file" | egrep -o "$skip" )
- if [ -z "$res" ]; then
- svn add "$file"
- fi
- fi
- done<<EOF
- $( svn st "$dir" )
- EOF
- exit 0
Download
PHP documentation browser
Even though I have my vacation right now I need to hack some ![]()
What I have done is a PHP documentation browser, I simply call it PHPDoc Browser, for GNU/Linux to read the PHP documentation locally on the computer (it’s a little bit like CHM for Windows). This little app isn’t that necessary but I thought it was a great thing for learning new stuff: The application is written in Mono/C# and uses SQLite for storing the search index.
Difference from CHM
Although there’s a CHM equivalent, XCHM, for Unix like systems I decided that my application serves a purpose: What I like about the PHP manual is that you can hit www.php.net/the_function in you browser and you will get the documentation for “the_function”. Of course I implemented the same functionality in PHPDoc Browser. You can also do a wild card search like “array_*” and you will get a list of all “array_*” functions.
I also implemented a free text search but that’s not “a real” FTS at the moment. FTS in SQLite in a little bit harder than in MySQL so while awaiting the next SQLite version the FTS is a bit of a hack. But you can quote the search to narrow it down
What’s learned?
Most of the unnecessary stuff I do I do to learn and the PHPDoc Browser is no exception. I learned how to use SQLite in Mono/C# and I also wrote a self contained installer in Bash, and Bash I havn’t really touch although I’ve been using GNU/Linux for 6-7 years! I found it quite fun writing the installer
Howto
If you would like to try it out:
- Download the install script
- Un-tar it (`tar zxvf phpdocbrowser-installer.tgz`)
- Make sure the script is executable (`chmod +x phpdocbrowser-installer`)
- Open a console and run the installer:
./phpdocbrowser-installer - Answer the questions and you’r ready to go.
The script will create a directory, ./phpdocbrowser-installer, in your home directory in wich you will find ./phpdocbrowser-installer, ./phpdocbrowser-installer and a directory, ./phpdocbrowser-installer, with the PHP documentation. A “run script” will also be installed either in ./phpdocbrowser-installer or in your home directory depending on how you answered the questions and if you agreed to create a desktop shortcut one will hopefully appear on your desktop (havn’t tested this on KDE but it should work fine in Gnome).
To run the application hit the desktop shortcut, if one was created, or invoke ./phpdocbrowser-installer from a console.
If you wish to remove the PHPDoc Browser run the following from a console:
./phpdocbrowser-installer.
And that’s that!
Download
PHPDoc Browser 17:31, Sat 17 October 2009 :: 8.9 MB
Screen shots
Updated Roxen trim tag
Jonas Walldén, CTO at Roxen, has improved my RXML tag contribution <trim></trim>. The code is now really beautiful and you can tell the difference from the code written by a lamer, as myself, and a real programmer, as Jonas. Jonas told me that the <trim></trim> tag will be part of the next Roxen release
Download
Roxen trim tag 17:31, Sat 17 October 2009 :: 4.8 kB
Pike trim module 17:31, Sat 17 October 2009 :: 4 kB
A Pike syntax highlighting module
So I thought I should try to port my syntax highlighting script, Syntaxer, written in PHP to Pike. Mostly for the fun of it but also to improve my knowledge of string handling in Pike. The greatest concern here is that PHP is a dynamic language and Pike is not (in the same sense) and the PHP version of Syntaxer heavily depends on dynamic loading of PHP files. The reason for this is that I generate the “syntax maps” dynamically from syntax files of Edit+. That means that if you want support for a new language just drop a .stx file in the right location and there you go. My script will convert that into a static PHP file, so that the conversion only needs to be done once, and load that file on the fly when that particular language is requested.
I thought that this method would be hard to implement in Pike – although it might be possible – so I had to come up with a slightly different approach. Frankly; it’s not that often you alter the .stx files or implement support for new languages so my solution is to manually create definitions for what ever language. But I still use the .stx files from Edit+ although one needs to copy and paste bit.
In the Pike solution each language is its own class that inherits the master class .stx. The only thing you pretty much need to put in the derived class is some .stx, .stx and .stx that specify what is what in the language. For example, the C++ definition looks like this:
- inherit .Hilite;
- public string title = "C++";
- //| Override the keywords mapping
- private mapping(string:multiset(string)) keywords = ([
- "keywords" : (<
- "auto","bool","break","case","catch","char","cerr","cin",
- "class","const","continue","cout","default","delete","do",
- "double","else","enum","explicit","extern","float","for",
- "friend","goto","if","inline","int","long","namespace","new",
- "operator","private","protected","public","register","return",
- "short","signed","sizeof","static","struct","switch","template",
- "this","throw","try","typedef","union","unsigned","virtual",
- "void","volatile","while","__asm","__fastcall","__based",
- "__cdecl","__pascal","__inline","__multiple_inheritance",
- "__single_inheritance" >),
- "compiler" : (<
- "define","error","include","elif","if","line","else","ifdef","pragma" >)
- ]);
- //| Override the default since # is no line comment in C++
- protected array(string) linecomments = ({ "//" });
- void create()
- {
- ::create();
- colors += ([ "compiler" : "#060" ]);
- styles += ([ "compiler" : ({ "<b>", "</b>" }) ]);
- }
And you really don’t need to make it more fancy than that. For most C-based languages the definitions in the master class .stx is enough. Just add the keywords to the .stx mapping and it looks better than nothing
HTML parser
One thing that differs from the PHP version of Syntaxer is that SGML-based, or tag based, languages will be run through a HTML-parser. The downside of the PHP version is that tag content will be highlighted as well, which of course isn’t what we want, but since Pike has a decent HTML parser that behaves like a SAX parser so I wrote a class, .stx, that uses that for highlighting tag based stuff. The .stx class also inherits .stx so the methods and members are the same.
I wonder why there’s no, built-in, HTML parser for PHP?
A Roxen tag module
Of course I had to write a Roxen tag module so that we can highlight source code in Roxen web pages. This was the reason for writing the Pike module at all. The tag is named .stx which might not be the most innovative name but what the heck! The beauty of it is that I made it possible, in the module settings tab, to create a surrounding HTML template for the output. When you run some code through the parser you get the highlighted source code as well as the name of the language and how many lines of code was highlighted and it might be nice to present that as well (just like the code blocks on this site). It’s tedious writing that surrounding HTML every time so now it’s just to put that in the settings and the code blocks will always look the same.
Finally
There’s some stuff left to do but the code works well enough to be usable. And I must say that the speed of the Pike version is like a thousand times faster than the PHP version!
Oh, and I have implemented support for the following language:
- ActionScript
- C
- C++
- C#
- CSS
- Java
- JavaScript
- HTML
- Perl
- PHP
- Pike
- Python
- Ruby
- RXML
- XSL
And that’s that for now.
Codify RXML tag and Syntaxer.pmod 17:31, Sat 17 October 2009 :: 183.9 kB
A Pike module and a Roxen tag
When you’r used to one programming language and start learning a new one you sometimes miss some features from the former. That happened to me a couple of years ago when we started using Roxen at work. What I missed was the trim functions from PHP. Sure, Pike and RXML can trim strings from whitespace but the beauty of trim, trim and trim in PHP is that you can trim characters as well as whitespace. And to my knowledge there’s no equivalent to trim and trim – that is only trim the left and right side respectively of the string.
So I wrote a Pike module that did that and also created an RXML tag of it. Just yesterday I rewrote that code since I do think I’ve become a better programmer and have come to learn Pike a lot better since my first try.
And I also wrote a method to shorten a string from the center and out, i.e. trim.
So here are some examples of usage:
- import .trim;
- string path = "/this/is/a/path/";
- write("%s\\n", rtrim(path, "/"));
- // Will output: /this/is/a/path
- write("%s\\n", trim(path, "/"));
- // Will output: this/is/a/path
- string long_str = "This is some string that's too long for us";
- write("%s\\n", ctrim(long_str, 20));
- // Will output: This is...for us
And here’s the RXML implementation:
- <trim right="" char="/">/this/is/a/path/</trim>
- <!-- /this/is/a/path -->
- <trim center="" length="20">This is some string that's too long for us</trim>
- <!-- This is...for us -->
And that’s that.
Trim Pike module 17:31, Sat 17 October 2009 :: 4 kB
Trim Roxen tag module 17:31, Sat 17 October 2009 :: 4.8 kB
Beautiful Roxen
I’ve been really busy lately at work with various things – developing a blogging system for starters. Most of the stuff you need for a blogging system is already available in Roxen CMS that we use at work. But one thing I needed, that isn’t available, was some commenting functionality. This is something that you could quite easily implement in the templates – Roxen uses XSLT for the templating system, or their own extension of XSLT so that you can write RXML code in the XSL templates.
RXML handles database driven things quite alright but if you develop things in this layer, the presentation layer, it gets harder to maintain or implement on a different server. What you can do then is writing a Roxen Module in Pike. You can create different types of modules – provider modules, RXML tag modules and so on – and you can also combine two or more modules into one. All RXML tags in Roxen is just wrappers for underlying Pike code, so you hide the logic in Pike and provide simple interfaces in RXML. First, let me give an example of how you loop over a database record set in RXML:
- <emit source="sql" host="myhost" query="SELECT name, email FROM employees">
- <a href="mailto:&_.email;">&_.name;</a>
- </emit><else>
- No records found
- </else>
It doesn’t get more simple than that! Another thing that’s great about the modules is that you can run arbitrary functions when you load the module which means that you can create databases and database tables when you first load the module. Roxen runs on MySQL internally and Roxen provide API:s to the internal MySQL database. What this means is that if you create database driven modules it’s just to install the module and the database and its tables will be created. No fuss!
Roxen, without the CMS, is a standalone open source web server and support the same kind of modules as the CMS does, but there’s more to the CMS!
The CMS modules
I’ve been poking around in the Roxen CMS source code for a while now and I have written some pure RXML tag modules – which is quite simple – but now I saw the opportunity to extend my knowledge and make a real CMS module. Since the CMS provide access control you don’t want emit tags to list stuff that the user don’t have access to. For instance: There’s a emit tag that you can use to list the most recent pages for a given path or pats. For example:
- <h2>Latest news</h2>
- <ul class='newslist'>
- <emit source="site-news" unique-paths="" maxrows="20"
- path="/comp-a/news/*/*.xml,/comp-b/news/*/*.xml,/secret/*/*.xml"
- >
- <li><a href='&_.path;'>
- <date type='iso' date='' iso-time='&_.published;' /> |
- <span>&_.title;</span>
- </a></li>
- </emit>
- </ul>
Now, you don’t want unauthorized users to see the items from the emit path do you. This tag in particular won’t show them either and I wanted the same kind of functionality in my set of emit tags. So I poked some more!
Another thing: Each file in Roxen CMS is version controlled through CVS and I needed to get some information about the first published version so I needed to find out how to access the CVS logs. The CVS thing also means that the files in Roxen CMS is saved to a virtual file system with an actual path, i.e emit, and what if there are comments to a page and that page gets moved? Thankfully Roxen CMS provides an API to run your own code when some kind of action takes place on a particular file – if it get moved, edited, deleted, purged and so forth. And I came to understand that I needed to poke around even more
Anyway; the more you dig around at the dark side of Roxen the more impressed you get by how clean the code is, how nice the API:s are and so on. I’m no Roxen beginner by now but up until now I haven’t really used the API:s to the internal workings of Roxen and I must say that after all, it was a bliss.
The comments module
So what will all this babbling lead up to? Well, what my module now does is:
- Upon load it creates an internal database with the table in which to store the comments. Here you can also make a dump or load a dump of the comments DB. In the settings you can set the default value for how long commenting should be enabled – lets say that after 30 days no further commenting should be allowed. You can also set a default author name if you allow anonymous commenting.
- It listens to the
emitwhich means that when actions takes place on web pages in the CMS the module will be notified and take actions if necessary. If the pageemithas comments to it and the page get moved the paths in the comments table will be updated to the new path so that comments always stick to its page even if the page is moved. Also, if a page gets deleted so will the comments. - It provides a set of RXML tags, one for listing comments – either to the page to which they belong or through globs so that you can make lists of “latest comments”, one for listing a particular comment for lets say editing, one tag for adding, one for updating, one for deleting a comment, one
emittag for checking if a user have admin permission to a given comment and oneemittag to check if the commenting form has expired or not and last one tag for counting the comments for a given page. - Not the least: I followed through and really documented, through Roxens module documentation functionality, the usage of the RXML tags
So I think that it covers what you’d expect a “commenting system” should be capable of doing. There’s may be some small things to solve as always when you develop stuff but it seems to work quite satisfying.
For anyone curious, here’s the source code:
Comments – Roxen CMS Module 17:31, Sat 17 October 2009 :: 36.3 kB
Here’s some screen dumps
Syntaxer 2.0.2 released
A few bugs was fixed in this release. I also noted a few new ones but that was mostly in the SyntaxMap class which gererates PHP arrays from the SyntaxMap files.
Changelog for this version
- Fixed a bug where the end of strings (quotes) wasn’t found correctly in languages that has no escape character (like XSL, XML and HTML).
- Also changed the Syntaxer::AutoDetect() method to return the extension of a file if no alias was found. In this way we can pass a path to the method and use the result directly as argument to `Sytaxer::__construct()`
- $file = '/some/path/to/file.xsl';
- $lang = Syntaxer::AutoDetect($file);
- $stx = new Syntaxer($lang);
- $stx->Parse(file_get_contents($file));
The Syntaxer can be read about and downloaded over here.
Database abstraction layer
I can’t remember the last time I made a web site that wasn’t hooked up to a database in one way or another! There’s nothing wrong with using the database functions that PHP provide directly but if you work with different types of databases – MySQL, Postgres, Oracle, SQL Server, SQLite and so on – you need to remeber how the functions for these databases work, in what order to pass arguments and what not. And maybe you write an application that teoretically could use an abritrary database and not MySQL that you developed the application for, and then it would be nice to use the same function calls no matter what database is being used.
This is where the database abstraction layer comes in handy. There already exist many of them so, again and as always, why bother writing another one? Well, it’s fun and you always learn new stuff and, for good and bad, third party APIs tend to be packed with features that you never use so it get quite tedious learning the API.
So I wrote my own database abstraction layer that only contain the functionality I mostly use. If I need more functionaly it’s easy to implement. When I wrote the abstraction layer I tried to evolve my OOP (object orientated programming) skills (which isn’t too great) so that in it self was I goal.
The abstraction layer contains of two interfaces and two master classes: A connection interface and class and a result interface and class. To create a driver for a new type of database you inherit the master classes and implement the interfaces. This is how the skelleton looks like:
- interface IDB
The connection interface - interface IDBResult
The query result interface - abstract class DB
The connection master class - abstract class DBResult
The query result master class
So a database driver should have two classes: One that extends the DB class and implements the IDB interface and one class that extends the DBResult class and implements the IDBResult interface.
Example
Here’s a simple example of usage:
- <?php
- require_once 'Database.php';
- //! MySQL example
- try {
- $category = $_GET['category'];
- $db = DB::Create('MySQL', 'localhost', 'uname', 'pw0rd', 'mydatabase');
- $db->Connect();
- $res = $db->Query("SELECT * FROM table WHERE cat_id = '%d'", $category);
- if ($res->NumRows() > 0) {
- while ($row = $res->Fetch()) {
- echo "Title: " . $row->title . "<br/>";
- }
- }
- }
- //! Could be DBDriverNotFound, DBConnectionFail, DBNotFound, DBQueryFail
- catch (Exception $e) {
- die("Error: " . $e->getMessage();
- }
- //! SQLite example
- try {
- $category = $_GET['category'];
- $db = DB::Create('SQLite', 'mysqlitedb');
- $db->Connect();
- $res = $db->Query("SELECT * FROM table WHERE cat_id = '%d'", $category);
- if ($res->NumRows() > 0) {
- while ($row = $res->Fetch()) {
- echo "Title: " . $row->title . "<br/>";
- }
- }
- }
- //! Could be DBDriverNotFound, DBConnectionFail, DBNotFound, DBQueryFail
- catch (Exception $e) {
- die("Error: " . $e->getMessage();
- }
- ?>
As you can see here we’re using exactly the same code, except for the instantiation of the database object, wether we’re using MySQL or SQLite, and that’s the meaning of the database abstraction layer.
Sources
I currently have two implementations for MySQL and SQLite. The MySQL implementation is what I’m using for this blog and the SQLite implementation is not very well tested yet.
The classes are pretty well documented and the documentation is bundled in the package below.
Database abstraction layer 17:31, Sat 17 October 2009 :: 42.1 kB
Get mimetype for a file
Sometimes you need/want to find out what mimetype a particular file has. If you have access to the PEAR “Fileinfo Functions” you can use them, but if you don’t here’s a simple solution that might be helpful.
NOTE! This is not as reliable as the Fileinfo Functions but it works in most cases.
In most GNU/Linux systems (if not all) there’s a file called mime.types (in /etc/) which is mapping of mimetype -> extension. The file looks like this:
- application/x-gnumeric gnumeric
- application/x-go-sgf sgf
- application/x-graphing-calculator gcf
- application/x-gtar gtar tgz taz
- application/x-hdf hdf
- application/x-httpd-php phtml pht php
Quite obvious you could build an array from this file where the mimetype could be the key and the extension(s) the value as an array:
- $_MIMETYPES = array(
- "mime/type" => array("ext1", "ext2", "ext3")
- );
When we’ve got this array populated we can easily look up a mimetype for an extention (and vice versa) by these two functions:
- /**
- * Find mimetype for extension
- * @global array $_MIMETYPES
- * @param string $ext
- * @return string|bool
- */
- function find_mimetype($ext)
- {
- global $_MIMETYPES;
- foreach ($_MIMETYPES as $mime => $exts)
- if (in_array($ext, $exts))
- return $mime;
- return false;
- }
- /**
- * Get extension(s) for a given mimetype
- * @global array $_MIMETYPES
- * @param string $mimetype
- * @return array
- */
- function findextensionfor_mimetype($mimetype)
- {
- global $_MIMETYPES;
- return isset($MIMETYPES[$mimetype]) ? $MIMETYPES[$mimetype] : array();
- }
The script to convert the mime.types database into an array an auto generate the two functions above looks like this (note that this PHP script is intended to run as a command line (cli) script, thus the shebang (#!/usr/bin/php), since we don’t need to do this conversion run time but only once):
- #!/usr/bin/php
- <?php
- /**
- * mimetyper.php
- * Turn the /etc/mime.types database into a PHP array
- * @author Pontus Ostlund <spam@poppa.se>
- */
- //! Mimetype file
- $mime_db = '/etc/mime.types';
- //! Resulting file for the PHP array
- $php_file = 'mimetypes.php';
- fileexists($mimedb) or die("No mimetype database ($mime_db) found\\n");
- $lines = @file($mimedb) or die("Couldn't read $mimedb");
- $fh = @fopen($phpfile, "w+") or die("Couldn't create $phpfile");
- $date = strftime("%A %B %d %Y", time());
- //! Write the PHP header to the file.
- @fwrite($fh,
- "<?php
- /
- * Find mimetype for file extension
- * Generated $date by " . basename(<strong>FILE</strong>) . "
- */
- $_MIMETYPES = array(n"
- );
- //! Loop through the lines array
- while ($line = array_shift($lines))
- {
- //! Match a comment or a blank line
- if ($line{0} == "#" || $line{0} == "\\n")
- continue;
- //! A line looks like:
- //! mime.type extention[ extension[ extension]]
- //! so lets grab the mimetype and extensions
- preg_match('/^(.?)\\s+(.?)$/', trim($line), $match);
- list(, $mime, $ext) = $match;
- if (empty($mime)) continue;
- //! This converts "ext ext ext" to "ext','ext','ext"
- $ext = join("', '", explode(' ', $ext));
- //! And lets write the new array index to the file
- fwrite($fh, sprintf(
- "\\t'%s' => array('%s')%s\\n",
- $mime, $ext, (empty($lines) ? "" : ","))
- );
- }
- //! Close the array, add a nifty search function, and close the PHP block
- //! and we're done!
- fwrite($fh, ");
- /**
- * Find mimetype for extension
- * @global array $_MIMETYPES
- * @param string $ext
- * @return string|bool
- */
- function find_mimetype($ext)
- {
- global $_MIMETYPES;
- foreach ($_MIMETYPES as $mime => $exts)
- if (in_array($ext, $exts))
- return $mime;
- return false;
- }
- /**
- * Get extension(s) for a given mimetype
- * @global array $_MIMETYPES
- * @param string $mimetype
- * @return array
- */
- function findextensionfor_mimetype($mimetype)
- {
- global $_MIMETYPES;
- return isset($MIMETYPES[$mimetype]) ? $MIMETYPES[$mimetype] : array();
- }
- ?>");
- fclose($fh);
- ?>
And this should result in a file called mimetypes.php that you just can include in you PHP scripts.













