×

PHP Beyond the web

Table of Contents

Welcome

About the author

I’ve been programming in PHP since late 2000. Initially it wasn’t by choice as my preferred languages at the time were Perl and Delphi (also known as Object Pascal). Things began to change after graduating from the University of Leeds with a degree in Computer Science in 1999 and starting out in a career as a freelance web developer. After only a couple of months I was offered the opportunity to take over a (relatively speaking) substantial government website contract from a friend who was exiting the freelance world for the safer and saner world of full-time employment. The only catch was that several thousand lines of code had already been written, and they were written in a relatively new language called “PHP”. Oh, and the only other catch was that I had about a week to learn it before taking over the site. So, as was the way at the time, I popped down to the local Waterstones bookshop (for the younger among you that’s where we used to get books. Books made of paper. And we had to go out and get them. Or order online and wait for days for them to be delivered. Did I mention they were made of paper?). With my paper copy of “The Generic Beginners Complete Guide to PHP and MySQL for Dummies Compendium” book in hand (I may not have recalled the title completely correctly) I settled down with a pint of ale (I’m in Yorkshire at this point, remember) and set about reading it. A few days later I was coding like a pro (well, stuff was working) and 12 years later I haven’t looked back. After a varied career as a freelancer and starting up a couple of, er, start-ups (IT related and not) with varying (usually dismal) success, I’m now a full-time programmer at The University of Oxford. My day job involves medium-scale data acquisition and management, statistical analysis and providing user interfaces for researchers and the public. The majority of my development work is done in PHP, either developing new projects or gluing together other peoples software, systems and databases.

Through-out my career I’ve always used PHP for web development, but for Desktop GUI work I initially used Delphi (and then Free-Pascal/Lazarus), complimented with bash shell scripting for other tasks. This was mainly due to learning them while at University. However, as PHP has matured I’ve increasingly used it beyond the web, and now I rarely use anything else for any programming or scripting task I encounter. Having been immersed in other languages like C++, Javascript, Fortran, Lisp (and probably others that my brain has chosen deliberately not to remember) by necessity during University and in some of my freelance jobs, I can honestly say that PHP is now my language of choice, rather than of necessity.

When I’m not tied to a computer, I would like to say I have lots of varied and interesting hobbies. I used to have. I could write a whole book about where I’ve been and what I’ve done and I’d like to think it’s made me a well rounded person. But these days I don’t. In large part this is due to the demands of my three gorgeous young daughters Ellie, Izzy and Indy, my gorgeous wife Parv and my even more gorgeous cat Mia. And I wouldn’t have it any other way. That’s what I tell myself, anyway…

Acknowledgements

Isaac Newton said “If I have seen further it is by standing on the shoulders of giants”. This book builds on, and hopefully adds to, the work of many others. The most notable of whom I would like to acknowledge below.

The authors of, and contributors to, the official PHP manual
An invaluable reference for PHP functions and syntax, to which I referred frequently during writing this book, both for fact checking and as an aide-memoir. Thanks!

The collective PHP wisdom of the Internet
For over 12 years I’ve used you for learning, research, play and profit. Too many sites & too many people to list here, if you’ve written about PHP on web then you may well be one of them. Thanks!

My Family
For allowing me a modicum of time to write this book, and supporting me unconditionally in everything I do. Usually. If I ask first. And there’s not something more important going on.

1 Introduction

If your e-reader has bought you directly to this page, then it has (perhaps wisely) skipped the books “front matter”, containing the front cover, table of contents, about-the-author section, acknowledgements and a few more bits and bobs. If that interests you, then scroll backwards. Otherwise, read on to start your journey through the wider world of PHP software.

1.1 “Use PHP? We’re not building a website, you know!”.

Both it’s current formal recursive moniker (“PHP : PHP HyperText Preprocessor”), as well as the name originally bestowed upon it by it’s creator Rasmus Lerdorf (“PHP : Personal Home Page”) reinforce the widely held view that PHP is a scripting language for the web. And that was true, back in 1995 when PHP was first created, and for a number of years afterwards. In the web arena PHP excels : it’s easy to use, quick to develop in, widely deployed, tightly integrated into web stacks (it’s usually the P in LAMP,WAMP,MAMP etc.), and of course because it is free and open source.

But many people don’t realise (or haven’t noticed, or choose not to notice) that PHP has evolved and grown up. It now closely resembles a modern, general purpose programming language. This lack of recognition is partly PHP’s own fault as it took a long time to get some of the fundamentals in place, such as OOP language constructs and even the ability to run without a web server being involved. However the programming community hasn’t helped; many programmers had a hard time seeing the potential for PHP to bring its rapid dynamic development model out of the web and into the wider computing environment, and many simply stuck with the “web scripting for beginners” dogma that was only really true of the early years.

Recent releases in particular have brought mainstream language features (e.g. closures, traits, better language support, namespaces & late static binding among many others) to the table. Performance has jumped up and up and up, memory usage (a bugbear of older versions) has dropped considerably, and PHP is now one of the more lean popular scripting languages. For even higher performance needs PHP now has a built in Op-code cache which is useful for oft-used scripts, and Facebook (the biggest user of PHP around) are one of several groups who are releasing alternative interpreters/VM’s with JIT compilers, leading to performance increases of up to 6 times that achievable with the standard Zend engine. And of course PHP core language development continues apace, version 5.5 has recently been released introducing more modern features like Generators, indicative that reports of PHP’s demise are quite premature!

If you’re a PHP web developer looking to work beyond the web, there has never been a better time to leverage your existing skill set and try your hand at desktop and system coding in PHP. If you’re a new programmer pondering which language to learn you can pick up skills to use in a wider range of scenarios than many other similar languages.

But why, why, oh why do it in PHP? Why not whip up a shell script? Why not learn C++ or another language typically used for software projects? The truth is these are valid options, and life may well work out just fine for you. But why turn down the opportunity to use your existing skills? Why not use PHPs integrated database access, reuse existing code and data from your web projects, take advantage of PHPs easy to use network libraries and functions, wallow in flexible text and data handling, and mix shell commands and other languages into PHP as needed to get best of both worlds? In short, why the heck not?

The aim of this book is to give you, the PHP coder, an insight into using PHP beyond the web. We’ll look at building desktop apps, command line tools, system daemons, network servers and other native applications. Importantly I’ll show you how you can do all of this without leaving the comfortable world of PHP and its friends.

Hopefully I’ve sold you on the story that PHP is a cross-platform, rapid-development focused, versatile language and is ideal for many different types of software. But if you still doubt the power of PHP to deliver real, non-web, software, and don’t want to read the whole book to find out what is achievable, then skip to Appendix D at the rear of the book to find examples of projects such as web servers, database engines, machine learning tools and many others all written in PHP. If they don’t inspire you to create your software in PHP, then maybe nothing will!

Further Reading

“PHP is much better than you think” - An article by Fabien Pontencier outlining the positives of PHP development and talking about the changes in the PHP ecosystem
http://fabien.potencier.org/article/64/php-is-much-better-than-you-think

1.2 Are you new to PHP?

This book is aimed at showing PHP developers how they can use their existing skills to write software instead of web pages. However I appreciate that some readers may be new to PHP and are reading this book to get a feel for what PHP is capable of. If you’re already a programmer, the comprehensive official PHP manual may be the best place to begin to get a feel for the differences between PHP and the languages you are used to.

If you’re not already a programmer, there are numerous “beginning PHP” books available from your favourite e-book retailer. In either case, the wider web also provides its usual breadth of in-depth knowledge and tutorials, just a judicious google away.

And finally, newcomer or experienced programmer, if you’re thinking of getting serious about PHP, it may be worth having a look at some PHP “best practices” websites before diving headlong into coding. It may save you a lot of trouble in the long run.

Further Reading

The Official PHP Manual
http://www.php.net/manual

A free online PHP course for beginners
http://ureddit.com/class/55471/programming-in-php

“PHP The Right Way” - A quick reference for PHP best practices, accepted coding standards, and links to authoritative tutorials
http://www.phptherightway.com/

“PHP Best Practices” - A short, practical guide for common and confusing PHP tasks
http://phpbestpractices.org/

“PHP Study Guide” - A PHP Study Guide aimed at those wanting to pass the ZCE (Zend Certified Engineer) exam
http://php-guide.evercodelab.com/

“PSR-What?” - A guide to the “PSR” PHP Standards Recommendations
http://www.lornajane.net/posts/2013/psr-what

1.3 Reader prerequisites. Or, what this book isn’t

To make the most of this book, you should have basic experience of programming in PHP (most likely for the web), a general programming or IT background, and a willingness to learn and be taken outside of your comfort zone.

This book isn’t an introduction to PHP or programming in general. Although you don’t need a computing degree or knowledge of advanced programming concepts, the book is hopefully pitched to the level of an average PHP programmer (one who has explored more than the very basics of PHP) and tries to explain any necessary concepts as we go along. It is also useful for advanced programmers who may choose to use it as a quick reference for exploring PHP beyond the web.

The book gives code samples to illustrate concepts and usage of functions, but generally doesn’t provide complete scripts or programs. It is written as a reference and introduction, to give you information and inspiration to help you develop your own PHP solutions. Likewise, when we discuss external tools and third party libraries, I will generally give an overview and basic introduction or example code, and then link to the official documentation and online tutorials. There is little point in a book like this, which covers a general area, going into great depth and giving extensive coverage to a particular piece of software, particularly those which already have comprehensive documentation available. We will of course cover any relevant areas where the official or online documentation is lacking, and point you directly at the relevant sections where necessary.

1.4 An important note for Windows and Mac users

Most of the examples given in the book are run and tested under Linux (the latest Ubuntu Desktop/Server flavour in most cases). PHP runs on many operating systems, including MS Windows and Apple OS X, and code often runs in an identical manner. However there are, of course, differences due to the file system, OS, available libraries etc., but to cover these all in this book would not be practical. In addition, some features such as those reliant on POSIX standards aren’t (easily) available on some systems like Windows. As OS X was derived from a POSIX compliant operating system, you will likely find more of the code compatible than with Windows, but your mileage will likely still vary. A good source of information for OS specific issues is the official PHP online manual, and in particular the user comments at the bottom of each page. Where possible areas specific to these other OSes have also been covered, for instance we will look at how to access the Windows Registry.

1.5 About the sample code

As you’ll see throughout the book I mainly use “traditional” imperative/procedural PHP in my coding examples to keep things as simple as possible for coders of all abilities. This book isn’t designed to be a lesson in coding best practices or style, a guide to OOP programming or to push any kind of programming model or dogma. I also avoid the use of any code frameworks. Many frameworks are based around the web model and don’t always perform as intended in the kind of applications we’ll be looking at, although some do now have “console” or “cli” modules. MVC style frameworks can be useful when building GUI applications (indeed the MVC paradigm pre-dates the web considerably!), but because of the many different implementation details and styles we’ll stick to framework-less code here.

It should be clear from the plain, straight forward PHP code presented how it can be used or adapted to suit your own programming style, framework or model. We will be mainly looking at task specific implementation details, leaving the hot topic of programming itself to the many other books available.

The sample code also generally doesn’t try to implement any particularly useful programs. It is designed to be as minimal as possible, in order to be as clear as possible and outline the scaffolding needed for a particular type of task. For example, the sample code for creating daemons focuses on the mechanics of creating a background process and running an event loop. This is to keep the code clear, as a PHP programmer you will likely already know how to check your Twitter account for the latest Justin Beiber updates, and just want to know how to daemonise the process. The code aims to be something of a reference for general-purpose PHP programming, rather than a tutorial on programming itself.

All of the sample PHP source code in this book is available for you to use and do with as you please without limit or license. Use or abuse it as you see fit!

If you have trouble running or understanding the sample code, see the feedback section in the last chapter of this book for details on how to contact the author for help.

1.6 External resources

Through out the book we will point you in the direction of external tools, resources and information, in one of two ways :

A “toolbox” like this
   
  Toolboxes like this contain details of useful tools and software
   
  Main website : http://www.a-useful.tool/
   
  Main documentation and installation info : http://wiki.a-useful.tool/

Further Reading

Useful articles, tutorials and reference information will be presented in Further Reading sections like this
https://www.very-useful-info.book/

1.7 Book formats/versions available, and access to updates

The book is currently available in pdf, epub and mobi format, all included in the same price! The epub and mobi formats are best for reading on dedicated e-readers. Even if you are reading it on an e-reader, you may wish to download the PDF version as well for use on your computer when coding as there are many links to useful websites through-out this book, and it is also easier to copy and paste code directly from the book. Whichever versions you use, you will always get free access to all future updates to the e-book.

1.8 English. The Real English.

Just a quick note to say that, no, those aren’t spelling mistakes, it’s British English (you know, the one that kicked it all off) as your humble author hails from the other side of the pond. Deal with it, we spell things differently. Any genuine spelling mistakes you spot will be dismissed as part of an evolving language, and then quietly fixed while you’re not looking. # Getting away from the Web - the basics

This chapter has an overview of the basic steps involved in breaking free from the web with PHP. We’ll look at the technical steps involved, and also at the differences in programming practises and focus.

1.9 PHP without a web server

Most PHP programmers have used PHP strictly in a web-server environment. In such an environment PHP is a CGI/Fast GGI or server module, called and controlled by the HTTP server (usually Apache, IIS, Nginx or similar). The HTTP server receives a request for a PHP based web page and calls the PHP process to execute it, which usually returns output back to the HTTP server to be sent on to the end user.

There have been a number of attempts to create local applications (desktop apps, scripting systems) with PHP using a locally installed web server, with some success. However the downsides of using this web-type model for local fapps are numerous:

  • For the value they provide in these scenarios, web servers such as Apache are often very over specified and resource hungry
  • Unless properly locked down, a web server running locally introduces a point of access to your machine for malicious visitors from the outside world
  • HTTP is a verbose protocol and (arguably) ideally suited for the web, however it’s often overkill for local inter-process communication and adds another resource and complexity overhead to your application
  • Your application interface typically runs in a web browser, and therefore looks anything but local (and comes with additional support/upgrade headaches unless you install a browser specifically for your app)

PHP, as of version 5.4, includes a built-in web-server which removes some of the problems described above. However it was designed for local testing of PHP scripts designed to be deployed on a fully-fledged HTTP server such as Apache in a live environment. It is a route that can be explored for some local apps, particularly where you are wanting to run a private instance of a PHP app that already exists. However its stability, reliability and suitability for production-ready local apps is yet to be proven, and it still comes with the baggage of HTTP and web browsers.

Since version 4.3, PHP has had a golden nugget hidden up it’s sleeve which solves all of these problems. The PHP CLI SAPI (Command Line Interface Server Application Programming Interface), to give it it’s formal name, is essential a long way of saying “stand-alone PHP”. It cuts out the need for a web server and provides a stand alone PHP binary for you to interact with. For instance, typing the following at a shell prompt

1 ~$ php /home/myfiles/myprogram.php

will simply execute your myprogram.php file (which you write mostly like any other PHP script) and return any output back to the terminal (unless you tell it otherwise) instead of being sent to the web server (which doesn’t exist!).

In general, PHP scripts called directly using PHP CLI will behave in the same way as when called through a web server, although there are some differences. For instance, you won’t find any $_GET or $_POST arrays, and PHP won’t send any HTTP headers by default, as these concepts don’t mean much beyond the web. Default settings like max_execution_time are set to sensible values for local use (in this case to 0 so your scripts won’t time out), output buffering is turned off, and error messages are displayed in plain text rather than HTML.

The PHP CLI SAPI is what we will be using for most of the examples in this book as it designed with exactly the same motivations as this book, taking PHP beyond the web. Full details of how to install and use the CLI SAPI are given in the next chapter.

1.10 PHP versions - what’s yours?

PHP has supported the CLI SAPI since version 4.3.0, so many of the examples in this book will run on any PHP version since then. At the time of writing, the current version of PHP is 5.5.1 and the code in this book has been tested against this version. If you find that a particular function doesn’t appear to exist or work as expected, check the online PHP manual for that function. This shows which versions support which functions, and describe any breaking changes created by newer versions.

If you are using an older version, there are some very good reasons to upgrade.

  • Performance has increased markedly (and, correspondingly, resources used have decreased) in recent versions
  • Although security is not always as critical with non-web applications (see the discussion on security later in this chapter for caveats), the security enhancements and security related bug fixes in recent versions are essential if you’re handling data from external sources, and if you’re using the same version for web work as well
  • Modern language features are available, which can help your coding productivity, as well as helping others take your code more seriously

As with the web versions of PHP, you can compile your own version of the CLI SAPI if you find the need. If you want to include extensions not pre-packaged by your OS software repositories, remove non essential code for performance reasons, or use any of the other compile-time options that PHP supports then learning to “roll your own” version may be worthwhile. A starter guide to compiling and installing PHP and related extensions are covered in Appendix A.

1.11 A few good reasons NOT to do it in PHP

This book is focused on doing everything in PHP, and for 90% of tasks PHP does all that is asked of it. But before you commit to rewrite your world in PHP, there are a few very good reasons that you may want to consider carefully before jumping in.

  • High performance requirements

If you need very high performance, particularly on limited hardware, PHP may not be for you. It has come on leaps and bounds in recent versions and regularly trounces languages like Python and frameworks like Ruby-on-Rails in benchmarks. However at its heart PHP is an abstracted scripting language which is never going to get the same performance for some tasks as lower level languages like C, which are closer to the bare metal. Don’t write it off for all high-performance tasks though, if you’re looking at performance with an eye to costs, you may find that the cost of savings on developer time through speed of development in PHP outweighs the cost of the extra hardware you throw at it to get the performance. Take a look at Chapter 9 to learn more about increasing performance, and find out how you can interact with languages like C from PHP so that you can still get some of the benefits of RAD (Rapid Application Development) in PHP.

  • Don’t (necessarily) re-invent the wheel

You can write a web server in PHP (see Appendix D for examples), but won’t existing servers like Apache do what you need? For many or most infrastructure “itches”, software exists (written in many different languages) that will give you the “scratch” you need. Unless you really need other features, the time spent writing your own is usually going to be greater than the time learning and implementing an existing piece of software. Of course, for some people the converse is true. “Because I can” can be a valid reason, and writing software is always a valuable learning experience.

  • Keeping the source closed

PHP is a scripting language, and this means that if you are writing software to sell or deploy elsewhere, you will be revealing your source code to the recipients. Many (including your author) will argue that being “open-source”, even commercially, is a good thing. But if it’s not your bag you will need to go to greater lengths to protect your code. Source code obfuscater are available online which use various tricks to make your code hard (but not impossible) to read, and a number of PHP compilers are now available to produce binary programs (albeit with limitations in terms of syntax and extension coverage). But the main PHP project hasn’t expressed any intention to support code hiding or compiling, and so the viability of such methods long term is not certain. The business case for closed-source software is also not a done-deal in the medium to long term.

1.12 Thinking about security

You are reading a book sample. This rest of this section is available in the full book.

2 Understanding the CLI SAPI, and why you need to

As we mentioned in Chapter 2, most systems programming you do will involve using the PHP CLI SAPI. It’s therefore important to have a good grasp of how to use it, the options for configuring and running it, and how it differs from the web based SAPIs you are used to. Luckily the differences are minimal, and many are intuitive, although it’s still worth having a thorough read of this chapter as the CLI SAPI forms the basis upon which most of your code will run.

2.1 What’s different about the CLI SAPI?

The following is a list of the main differences in the CLI SAPI implementation :

You are reading a book sample. This rest of this section is available in the full book.

2.2 CLI SAPI installation

To use the PHP CLI SAPI, you may need to install it first. Appendix A gives details on installing (and compiling, where necessary) PHP. However you may find that it is already installed if you have PHP installed (often in a folder called sapi/cli in the PHP program folders), and if not it is usually available in modern OS software repository (e.g. in Ubuntu a package called php5-cli exists and can be installed from any package manager or via the command line with sudo apt-get install php5-cli). If it is installed in the command line search path, typing php -v on the command line will print the version details, confirming it is indeed installed.

2.3 PHP command line options

The PHP binary will accept a number of command line options/switches/arguments which affect its operation. A full list can be seen by typing php -h. Although some apply only to the CGI SAPI, the following are some of the more interesting and common ones used when interacting with the CLI SAPI:

  • -f or --file

This allows you to specify the filename of the script to be run, and is optional. The -f option exists to allow compatibility with software and scripts such as automation software which can programmatically call command line programs but require filename arguments to be formed in this way. It also allows default filetype handlers to be easily set on Windows for PHP scripts. The only real difference in usage between the two versions of the command above come when interpreting command line arguments passed to the script, which we look at in the “Command line arguments for your script” section below. In most cases, the two following lines are mostly equivalent :

1 ~$ php -f myscript.php
2 ~$ php myscript.php
  • -a or --interactive

Runs PHP interactively. This allows you to type in PHP code, line by line, rather than executing a saved PHP script. This mode of operation is often called a “REPL” (Read-Eval-Print-Loop). As well as providing an interactive interface for testing and developing code, it can also act as an enhanced PHP enabled shell or command-line, and in the next chapter on development tools we’ll look at this and third party PHP REPLs more closely.

  • -c or --php-ini

Specifies the PHP ini file that PHP will use for this application. This is particularly useful if you are also running web services using PHP on the same machine, as if it is not specified PHP will look in various default locations for php.ini and may end up using the same one as your web service. By providing one specifically for your CLI applications you can “open up” various restrictions that make more sense for offline applications. Note that by using the CLI SAPI, PHP will automatically override several php.ini settings regardless of whether you specify a custom ini file. These overridden settings are those that affect the behaviour outlined in the “Whats different about the CLI SAPI?” above, and while the php.ini file is ignored in these cases you can revert or change these settings directly in your code using ini_set() function or similar. You can also use the -d or –define option to set options (e.g. php -d max_execution_time=2000 myscript.php). If you are deploying software onto machines that you do not control (e.g. if you are selling software for users to install on their own machines), it makes sense to use one of these mechanisms to ensure that PHP will be running with the settings you expect, not the settings the user may happen to have. See -n below as well.

You are reading a book sample. This rest of this section is available in the full book.

2.4 Command line arguments for your script

As we’ve seen above, passing arguments to PHP itself is straightforward and done in the normal way. However passing arguments for use by your PHP script is a little more complicated, as PHP needs to know where it’s own arguments stop and where your script’s start. The best way to examine how PHP deals with this is through some examples. Consider the following PHP script:

1 <?
2 
3 echo "Number of arguments given :".$argc."\n";
4 
5 echo "List of arguments given :\n";
6 
7 print_r($argv);

There are two special variables in the script above :

  • $argc : This records the number of command line arguments passed to the script
  • $argv : This is an array of the actual arguments passed.

Lets save the script as arguments.php. Now lets call it as follows :

1 ~$ php -e arguments.php -i -b=big -l red white "and blue"

You will get the following output :

 1 Number of arguments given :7
 2 List of arguments given :
 3 Array
 4 (
 5     [0] => arguments.php
 6     [1] => -i
 7     [2] => -b=big
 8     [3] => -l
 9     [4] => red
10     [5] => white
11     [6] => and blue
12 )

As you can see, all of the arguments given from the filename onwards in the command are passed to the script. The first, -e, which is used by PHP itself is not passed through. So, as a general rule, everything after the filename is treated as an argument to the script, anything before the filename is treated as an argument for PHP itself, and the filename is shared between the two.

Of course there is an exception. As we learned above, as well as specifying the filename of your script on it’s own, we can also pass it as part of the -f flag. So if we execute the following command:

1 ~$ php -e -f arguments.php -i -b=big -l red white "and blue"

we get the following unexpected output :

 1 phpinfo()
 2 PHP Version => 5.4.6-1ubuntu1.3
 3 
 4 System => Linux dev-system 3.5.0-37-generic #58-Ubuntu SMP Mon Jul 8 22:10:28 U
 5 TC 2013 i686
 6 Build Date => Jul 15 2013 18:23:34
 7 Server API => Command Line Interface
 8 Virtual Directory Support => disabled
 9 Configuration File (php.ini) Path => /etc/php5/cli
10 <rest of output removed for brevity>

You may recognise this as the output of calling php -i. Rather than treating arguments after the filename as belonging to the script, PHP has treated the -f argument as one of it’s own, and continued to “own” all of the following arguments. As -i is a valid PHP argument, it decides that it was what you wanted and invokes its “information” mode. If you need to pass the filename as part of the -f flag rather than as an argument on it’s own, you will need to separate your scripts arguments using --. So, for the command above to work as expected, we need to alter it to read:

You are reading a book sample. This rest of this section is available in the full book.

2.5 The many ways to call PHP scripts

You are reading a book sample. This rest of this section is available in the full book.

2.5.1 From a file

You are reading a book sample. This rest of this section is available in the full book.

2.5.2 From a string

You are reading a book sample. This rest of this section is available in the full book.

2.5.3 From STDIN :

You are reading a book sample. This rest of this section is available in the full book.

2.5.4 As a self executing script : Unix/Linux

You are reading a book sample. This rest of this section is available in the full book.

2.5.5 As a self executing script : Windows

You are reading a book sample. This rest of this section is available in the full book.

2.5.6 Windows php-win.exe

You are reading a book sample. This rest of this section is available in the full book.

2.6 “Click to Run” your PHP

You are reading a book sample. This rest of this section is available in the full book.

2.6.1 Clickable icons : Linux

You are reading a book sample. This rest of this section is available in the full book.

2.6.2 Clickable icons : Windows

You are reading a book sample. This rest of this section is available in the full book.

2.6.3 Clickable icons : Ubuntu Unity

You are reading a book sample. This rest of this section is available in the full book.

2.7 Quitting your script

You are reading a book sample. This rest of this section is available in the full book.

3 Development tools

So far we’ve outline some of the basics of using PHP without a web server, but before we really get into the nitty-gritty details of developing useful non-web software we’re going to take a slight detour into the world of PHP development tools. This is for two reasons. Firstly, before deciding which techniques and methods you are going to use (Interactive CLI script or GUI interface) it is important to understand what tools are available to help support your choice of programming methodology. Secondly, you will often need to adopt a different workflow when developing a new type of software that you may not be familiar with, so you may need to find new tools to fit the new way of working.

In general, you can use the same development tools for general purpose programming as you do for web development, after all PHP syntax is the same wherever its executed. The main differentiator is your workflow, and you may find yourself being more productive with different tools.

Of course one of PHP’s dirty secrets is that many PHP programmers don’t use debuggers, unit testing, build systems or code profilers, and in many cases don’t even know what they are. Usually it’s because many PHP scripts are developed as small, uncomplicated, low-risk projects by single developers, and the overhead of learning and deploying extra tools is an unnecessary burden (particularly for those without a formal computer science or programming education). Don’t worry if you fall into this camp, we’ve all been there (and like to re-visit it from time to time!).

However when you start venturing into general purpose programming, you’ll find that the complexity of projects often increases. Rather than a selection of short lived scripts with limited shared state executing in less than a second, you’ll get into larger code bases with longer running processes and more complex dependencies. In such scenarios, the effects of bugs can be magnified and tracking them down becomes increasingly hard. Code is revamped less often and can be relied upon by an organisation and remain deployed for much longer, increasing the burden of maintaining code in the future. If you’re not a regular user of the type of tools described in this chapter, you might find this the ideal opportunity to delve into the wider world of PHP development tools, and give yourself a good foundation on which to start building real software.

We’ll look in turn at different classes of tool, and give some examples of, and links to, popular tools of each type.

3.1 PHP REPLs

When you want to test out a few lines of PHP, your default instinct may be to create a new PHP file, save it, and then execute it with PHP. There is a better, faster and more interactive, way however. The PHP “Interactive Shell”, also known as the PHP REPL (Read-Eval-Print-Loop), is a quick and easy way to type in code and have it execute immediately. Unlike executing single lines of code using php -r, the REPL (started by calling php -a) keeps the scripts state (e.g. contents of variables and objects) in-between each line that you type, until you exit. You can use all of PHP’s functions, although no libraries are loaded by default, and you can include() or require() existing files of PHP code. This latter capability is useful for debugging the final output of a problematic script; simply include() your script which will execute the script and, as long the script doesn’t terminate prematurely, then you can echo() or print_r() or otherwise explore the state of the variables and other resources at the end of the run. The following example is a capture of an actual interactive REPL session using the standard PHP REPL. Other brands of REPL are available, and are listed later in this section.

You are reading a book sample. This rest of this section is available in the full book.

3.2 Build systems

Build systems are used for building and deploying software to development and/or live systems. They automate many of the repetitive tasks that occur when deploying a new revision of a software system. Build scripts and build systems originated with languages like C which required compilation before the software was able to run, and covered steps such as compiling and linking various software modules as well as managing source code dependency issues. Although PHP is not a compiled language, a build system can still be a great time saver performing tasks like :

  • gathering and managing assets (images, data files etc.)
  • packaging or moving scripts and assets
  • archiving older versions
  • running automated tests and other quality control processes
  • running code minification or obfuscation processes
  • starting and stopping related services
  • refreshing and resetting your test environment
  • cleaning or initialising databases
  • documentation generation
  • team notification

or indeed anything else needed to ensure the latest version of your script is deployed successfully. As well as saving time, it also helps to ensure consistency and prevent mistakes. As you’ll no doubt know, trying to solve an obscure bug that occurred because you forgot to copy over the latest copy of a minor data file isn’t fun.

You can of course construct your own build system using shell scripting (or even better, PHP CLI scripts!), which may suffice for small or simple projects. For larger or more complex systems an existing build system may be a time saver. Many existing systems can cope with PHP, but one of the best for PHP work is Phing. Phing was built for PHP, and can be easily extended with PHP classes and has a wide range of PHP related tasks already built in. Telling Phing what to do is a matter of creating simple XML files, which you can even create programmatically using the PHP XMLwriter extension if you need to.

You are reading a book sample. This rest of this section is available in the full book.

3.3 Continuous Integration

You are reading a book sample. This rest of this section is available in the full book.

3.4 Debuggers

A debugger provides an easy way for you to inspect the internal state of your application, often while it runs, at fixed points in the code, or after a crash. This can give you valuable insight into bugs, particularly those in long running code or those relating to how external data affects variables, as well as direct flaws in your code. Debuggers can give you a “stack trace”, which shows the nested set of functions (or “stack”) you are currently in. They can also provide the current contents of variables, objects and their members, the state of resource identifiers and so on. Some additionally provide profiling capabilities (see the section on profilers later in this chapter).

There are a number of debuggers available for PHP, the most prolific (and arguably the most versatile) is Xdebug.

You are reading a book sample. This rest of this section is available in the full book.

3.5 Testing and Unit Testing

You are reading a book sample. This rest of this section is available in the full book.

3.6 Static code analysis

You are reading a book sample. This rest of this section is available in the full book.

3.7 Virtual development & testing environments

You are reading a book sample. This rest of this section is available in the full book.

3.8 Source/version control systems & code repositories

You are reading a book sample. This rest of this section is available in the full book.

3.9 IDEs and editors

You are reading a book sample. This rest of this section is available in the full book.

3.10 Documentation generators

You are reading a book sample. This rest of this section is available in the full book.

3.11 Profilers

You are reading a book sample. This rest of this section is available in the full book.

3.12 Other tools

You are reading a book sample. This rest of this section is available in the full book.

4 User facing software

After slugging through the preliminary information necessary to understand developing PHP in a non-web context, we’re now getting to the nitty-gritty of how to start communicating with our users without the rendering engine of a web browser.

Some software sits and happily runs without any interaction from humans, we’ll call this “system software” and will examine it in Chapter 6. However a large proportion of software requires interactivity with the user, and there are a number of different ways of performing the interaction in PHP. From interactive command lines through to fully fledged GUI (Graphical User Interface) based software, it can all be done with PHP.

When choosing how your software interacts with humans, you need focus on the needs of the user and put your own preferences to one side if at all possible. Command line interactions are (usually) very straight forward to code, text is, well, just text and it’s what PHP excels at. However many users, particularly non-technical users, shy away from text based CLI software. Unless you are meticulous about your software structure and interface flow, text based input can be :

  • prone to error
  • hard to navigate
  • require mental agility on the part of the user
  • have low discoverability
  • and be initially slower to use than a GUI

If you choose a text based interface, make sure your target users will be happy with it (you may be, but often your users aren’t much like you!). Text interfaces, because of the simplicity of coding, can often be a good choice for proof-of-concept and prototype software (particularly where the main benefits of the application are in what it does, not in how the user interacts with it), but be aware that temporary interfaces have an uncanny way of “sticking around”. Additionally, some non-technical commissioning users (the words “pointy-haired-boss” spring to mind here) can’t see past an interface to the application beneath and may not be happy with your proposed project, even after you explain that the interface is just temporary. This is understandable to some extent as, for most users, the application IS the interface. It is all they see and use day-in day-out. The changes to their files and other functions performed just happen by “magic” of course. Good user interface design (whether it be for text based interfaces or fully-fledged GUIs) is a field of endeavour in and of itself, so get help from the professionals where you can, read some of the multitude of books available on the subject, or take a course. If you ever looked at a piece of software and wondered how it got so popular while your preferred, much more functionally superior software, languished in obscurity, the answer is often at least partly down to good user interface design.

4.1 Command line interface basics

As described above, there are still uses for text-based interfaces, particularly in environments with technically-adept users. When creating a text based program to run on the command line, there are three primary considerations over-and-above the PHP you are already accustomed to. These are

  1. Getting keyboard input
  2. Outputting text (and graphics) to the screen
  3. Program flow control

Rather than describe each one in isolation, we will instead look through a simple program that contains elements of each. Read through the code and comments below. The program is a screen-saver like routine which fills the shell with colour via a wiggling snake-like cursor.

You are reading a book sample. This rest of this section is available in the full book.

4.2 Advanced command line input

You are reading a book sample. This rest of this section is available in the full book.

4.3 Using STDIN, STOUT & STDERR

You are reading a book sample. This rest of this section is available in the full book.

4.4 Partial GUI elements - dialogs

You are reading a book sample. This rest of this section is available in the full book.

4.5 Dialogs invoked from the shell

You are reading a book sample. This rest of this section is available in the full book.

4.6 Windows dialogs

You are reading a book sample. This rest of this section is available in the full book.

4.7 Static HTML output

You are reading a book sample. This rest of this section is available in the full book.

4.8 Complete graphical interfaces (GUIs)

Many users expect full graphical user interfaces (GUIs) these days, with very good reason. GUIs, when done right, are good for discoverability and ease of interaction. They can provide intuitive human-friendly ways of presenting data and information and are ideal for event based programming.

Most GUI programs are written to use “Widget Toolkits” (also known as GUI Toolkits). These are toolkits or frameworks that provide code for the building blocks of GUIs; windows, forms, buttons, lists, mouse interactions, menus, events and more. There are a large number of widget toolkits, some are low-level and platform specific such as the Windows API for Windows or Carbon for Mac OS X, some are higher level and cross platform like GTK+ and QT, and some are language specific, such as Swing for Java and LCL for Object Pascal. There is a fairly comprehensive list on Wikipedia.

Further Reading

Widget Toolkits on Wikipedia
http://en.wikipedia.org/wiki/List_of_widget_toolkits

Two approaches exist in PHP for using these toolkits to make graphical interfaces. The first is via direct bindings - your PHP script invokes the toolkit directly via a library or PHP extension. The second is via a helper application - your PHP script tells the helper application what it wants to display using an intermediate language like HTML or XUL, and the helper application deals with calling the necessary elements of the toolkit (or its own code) and displays the interface for you. The former is the traditional way of writing graphical applications, the latter is more akin to the web approach (where the browser is the helper application to which your scripts send HTML). Both approaches have pros and cons, below we will look at some of the current implementations of both methods.

4.9 Understanding GUI and event-based programming

It’s worth spending a few moments at this point talking about one of the main differences between GUI apps and, for instance, command line programming. Most GUI programming is event-based programming, which can sometimes take a while to get your head around if you’ve not programmed using this style before.

With a GUI program, you typically start programming “sequentially” as normal by opening a window or form and populating it with buttons, text, images, data or whatever.

At this stage your program then goes into a waiting loop, usually waiting for the user (or sometimes the system) to do something. That may be waiting for them to click a button, enter some text, select an option or wait for a new item of data to arrive. Each of these things is called an event, and you typically can’t predict in which order they will happen. So your code will bind these events to functions or methods, and call these as the relevant events occur.

This means that your code is often executed out of sequence, so keeping abreast of state becomes much more nuanced than in command line scripts and even web scripts, which run sequentially from top to bottom. The most common way of dealing with event based programs is to use Object-Oriented (OO) programming techniques, and indeed the rise of GUI software drove the rise in OO programming (or followed on from it, depending on who you talk to). Its still perfectly possible to use traditional “imperative” style programming of course, particularly if you are careful around issues such as state and scope, but if you intend to do a lot of GUI programming and aren’t familiar with OO, you may be doing yourself a favour if you pick up an OO primer and have a quick flick through.

Further Reading

Object-oriented Programming on Wikipedia
http://en.wikipedia.org/wiki/Object-oriented_programming

“Introduction to PHP OOP” by Lorna Mitchell
http://www.lornajane.net/posts/2012/introduction-to-php-oop

“Object-Oriented PHP for Beginners” by Jason Lengstorf
http://net.tutsplus.com/tutorials/php/object-oriented-php-for-beginners/

Web programming is of course event-based in a sense, you typically present an HTML page to the user and then wait for them to click a button or link, and handle that “event” with a different HTML page (or another run through of the same script with different parameters). Each of your HTML pages (or the PHP scripts that generate them) is in some-way equivalent to the individual functions or methods called in our GUI scripts, and your web server (Apache etc.) is equivalent to the waiting loop in our GUI program that holds things together and reacts to our events. The difference is that in web based programming, with PHP in particular, we operate on a shared-nothing (or almost nothing) basis - each script execution or HTML page fetch is deliberately separate from another unless we otherwise act to share some state. In GUI programming, we are typically operating from within the same script set under one process, and thus it is a share-(almost)-everything architecture. Once you’ve got your head around this concept, you will find it much easier to plan and map out your software.

Now lets look at some of the GUI toolkits available.

You are reading a book sample. This rest of this section is available in the full book.

4.10 PHP-GTK

You are reading a book sample. This rest of this section is available in the full book.

4.11 wxPHP

You are reading a book sample. This rest of this section is available in the full book.

4.12 Local web server & browser

You are reading a book sample. This rest of this section is available in the full book.

4.13 PHP’s Built-in testing server

You are reading a book sample. This rest of this section is available in the full book.

4.14 Web sockets & browser

You are reading a book sample. This rest of this section is available in the full book.

4.15 SiteFusion

You are reading a book sample. This rest of this section is available in the full book.

4.16 Winbinder

You are reading a book sample. This rest of this section is available in the full book.

4.17 Adobe AIR

You are reading a book sample. This rest of this section is available in the full book.

4.18 Titanium

You are reading a book sample. This rest of this section is available in the full book.

4.19 PHP-Qt

You are reading a book sample. This rest of this section is available in the full book.

4.20 PHP/TK

You are reading a book sample. This rest of this section is available in the full book.

5 System software

In the previous chapter we looked at user facing software, that is, software which the human user interacts with directly. In this chapter we are going to look at what I will term “system software”, software that does things (generally) without a human driving it.

To write system software we typically need to create a “daemon”, which is a continuously running process. Once we’ve created the Daemon, we need to decide its primary function. Some daemons work continuously (measuring, monitoring, communicating and so on), others wait and work in response to events (network connections, system changes, user actions). We will look at how to create a daemon in PHP and set it working continuously. We will then look at how we can take that basic daemon and instead of working continuously, make it to just spring into life to react to certain events that occur in our system.

In the user facing software we looked at in the previous chapter, the software is typically used by one person or process at a time. System software, on the other hand, often serves many different “clients” at the same time. A “client” in this context may be a network client (e.g. a web browser for a web server), a user process (a GUI client accessing your API server), a file (a log file for a logging server) or similar. While doing so it needs to remain “responsive”, that is one client shouldn’t have to wait while another client’s request finishes. Think how backed up the web would get if Apache could only serve one web page at at time! To manage concurrent tasks in PHP and maintain a responsive daemon, we can use task dispatch and management systems, which we will look at in the final part of this chapter.

5.1 Daemons in PHP

A daemon is a program that runs, usually continuously, as a background process. It often doesn’t interact with users directly, but performs background tasks or responds to system events or calls from other software, network requests or other machine-to-machine events. Examples of programs that run as daemons include Cron (which waits in the background and executes tasks based on the current time), and Apache (which sits and waits for calls from remote machines for web resources). Daemons usually :

  • run permanently (or for a long time, or until a predetermined event),
  • start up at boot time
  • perform useful tasks
  • are owned by root or a (non-human) system user.

However these criteria don’t universally apply to all daemons, in fact the only concrete thing daemons have in common is that they don’t have a controlling terminal (tty), and thus are deemed to be running “in the background”. Without a tty the software cannot get user input from the keyboard or display output back to the user, via the terminal (though there are other ways to directly and indirectly interact with a user). Although consuming minimal resources was traditionally a key trait of background processes, that is not now commonly the case. Software “servers” such as Database Management Systems and Web Servers run as background daemons but often consume large (or even all) of the system resources and often have machines dedicated just to running them. So it may be best to think of daemons as any permanently running software which doesn’t usually directly interact with the user.

5.2 Creating a daemon

To create a daemon in PHP we use the PHP process control extension, or PCNTL, which is only available on Unix/Linux type systems. Most pre-compiled versions of PHP include the PCNTL extension, but if not you will need to re-compile using the --enable-pcntl option, or install the extension using your package manager. See Appendix A for details.

The outline process for creating a daemon is as follows :

  • We run a process (PHP script). We will call this the parent process.
  • From the parent, we fork (copy) a child process.
  • The parent process then exits. The child process is now parent-less.
  • init adopts the parent-less child process. init is the original process started by the kernel when it boots and is the ancestor of all processes.
  • We then dissociate (detach the child) from the terminal we started the parent in. This is so that
    • any of our output doesn’t appear in the terminal
    • killing the terminal won’t kill our child process
    • we are truly running in the background
  • To dissociate (detach), we need to :
    • move the child process into it’s own POSIX process session
    • fork it once again (into the grandchild process) and kill the child process
    • close any file descriptors such as STDIN that may tie it to the terminal

Once all this is done, you will be returned to the command prompt in the terminal (assuming thats where you started your original parent process from) and your (grandchild) daemon will be running on its own. You will only be able to interact indirectly with your daemon from now on. Finally, assuming that the daemon is to run continuously (or for a set period of time) rather than just completing a task and exiting, the process will need to enter a loop where it will continuously cycle and, for instance, await events or perform continuous tasks. Its probably also wise to give it the ability to exit upon demand.

This may sound quite a long and involved process, however it is fairly straight forward in PHP. The following script follows this process and outlines the basics.

You are reading a book sample. This rest of this section is available in the full book.

5.3 Network daemons using libevent

In the previous section we used a fairly basic while(1) { } event loop to keep our daemon running and responding to events or doing useful work. The advantage of that approach is that is very simple for basic needs and is implemented natively in PHP with no external dependencies. The downside however is that it leaves you to implement all of the details, and the complexity increases as your project grows.

One popular alternative to consider is libevent, a library which provides a framework for dealing with event based programming. This library can be accessed in PHP though two different PECL modules :

  • pecl-libevent : This is an older module, and is fairly simple and straight forward to use. However it doesn’t support libevent2 (only 1.x versions, 1.4.0 or above.), and thus has fewer features
  • pecl-event : This is a complete rewrite of previous Pecl module of the same name abandoned in 2004. It is currently actively developed, and supports libevent2. This has more options including specific classes tailored for HTTP, DNS, SSL and other types of event connections. For these reasons this is the module we will use in the examples below.

Libevent describes itself as “… a library that provides a mechanism to execute a callback function when a specific event occurs on a file descriptor or after a time-out has been reached”. In layman’s terms, this means that libevent will execute a function of your choice either at pre-determined time intervals, or when a particular “file descriptor” event occurs. “File descriptors” in PHP cover not just events occurring on actual files, but anything that can be treated as a file or stream. This includes network sockets and system streams like STDIN. In fact, due to a problem with epoll (a Linux kernel event notification system, used by libevent) compatibility, libevent typically cannot be used for file events (detecting file accesses and modifications etc.) on many platforms. Because of this, we will additionally look at inotify in the next section for use with file events. Indeed, you should only really consider libevent for network/stream type events, which is where it really shines.

Libevent also offers event buffering, so in demanding environments it will queue events for you to process at your leisure, and you won’t risk missing something because your script was off doing something else. This is particularly important in a non-multitasking environment like PHP. Note that libevent just deals with responding to events, not creating a daemon in the first place, so you will still need to use code from the previous section to turn your script into a daemon before using Libevent to do the work.

The following example shows how we can use the pecl-event module to call libevent to act as a very simple http server. For brevity the following example runs as a standard CLI process, you can daemonise it using the techniques discussed in the previous section if you need to.

You are reading a book sample. This rest of this section is available in the full book.

5.4 File monitoring daemons using inotify

You are reading a book sample. This rest of this section is available in the full book.

5.4.1 Using the inotify PECL extension

You are reading a book sample. This rest of this section is available in the full book.

5.4.2 Using the inotifywait command

You are reading a book sample. This rest of this section is available in the full book.

5.4.3 Inotify limits

You are reading a book sample. This rest of this section is available in the full book.

5.5 Task dispatch & management systems

We’ve looked in this chapter at how you can fork new processes, and we’ll look in the next chapter at how to execute or call, and talk to, other external commands. These examples should give you some good ideas of how you can create worker tasks to carry out processing in parallel with your main PHP scripts. There are many good cases for rolling your own task/worker dispatch and management scripts, but there are also many times when it may be more prudent to use something already written by an expert. Luckily in these cases we can take advantage of any one of a number of excellent task dispatch and management systems that work with PHP. A few of the most common and useful systems are listed below, but first we’ll look at one particular system - Gearman - which has fantastic PHP bindings and good community support, and is an ideal task system to break your teeth in with.

5.6 Gearman and PHP

From the PHP manual :

  • “Gearman is a generic application framework for farming out work to multiple machines or processes. It allows applications to complete tasks in parallel, to load balance processing, and to call functions between languages. The framework can be used in a variety of applications, from high-availability web sites to the transport of database replication events.” http://www.php.net/manual/en/intro.gearman.php

In essence, Gearman is a “middle-man” between your main scripts and your worker scripts. Your main scripts can fire off tasks to Gearman, and Gearman will allocated those tasks to workers when they become available. It will then monitor the workers and report their progress back to the main script along with the results of the task. You can have multiple Gearman servers for redundancy, and you can operate everything in a distributed manner across many different machines. You simply tell your main scripts where your Gearman server(s) are, and start firing off tasks. You don’t even have to wait for the results of the tasks if you don’t want to, and you can register call-back functions to handle results from multiple tasks as they come in. To configure the worker scripts, you tell them where the Gearman server(s) are and specify which tasks that particular script can handle, and add a little code to feed back progress during the task if you wish. You then fire up your worker scripts and they sit and wait for tasks to come in.

To help you visualise the possibilities available with Gearman, the diagram below shows a multi-machine setup with redundant Gearman servers.

Example with multiple machines/masters/clients/servers

Example with multiple machines/masters/clients/servers

In this particular network, we have six PHP “master” scripts running on two different machines. Each of these masters are connected to two Gearman server instances, one running on its own machine and one on a shared machine. In turn, there are five PHP “worker” scripts, four running on their own machine and one on the shared machine. All of the worker scripts are also connected to both of the two Gearman server instances. In this set up, you can see that if one of the Gearman server instances are unavailable, the system would continue to run as all components are connected to both server instances at the same time, and so can use either. Likewise if one or more master or worker scripts are unavailable, work still happens as the Gearman server instances dish out the jobs to the remaining scripts. And the best thing of all is that Gearman takes care of this “fallback” system for you. You simply point your masters and workers at the Gearman servers and Gearman deals with routing the jobs appropriately.

You are reading a book sample. This rest of this section is available in the full book.

5.7 Other task dispatch systems

You are reading a book sample. This rest of this section is available in the full book.

6 Interacting with other software

Few pieces of software run in total isolation, most will talk to various other processes and programs on your system. Particularly on open source systems (where licensing is less of an issue) there is often little point re-inventing the wheel when, with minimal effort, you can simply call upon other existing software and libraries to perform tasks that are already a “solved problem”.

Although you may have come across some of these methods when programming for the web, there is generally less call for interactions when your script will execute and be gone in the blink of an eye. There are a number of different methods for invoking and interacting with other software, which we will cover below. The primary use for these methods is interacting with non-PHP software, however by virtue of the fact that these methods are language agnostic they can of course be used for communication between two or more scripts written in PHP.

6.1 Starting external processes from PHP, or “shelling out”

Before we can start talking to another process, that other process has to start running. You can of course manually start another piece software at the same time as your PHP script, but often it is useful for the PHP script itself to start the other software when it needs to. This is often referred to as “shelling out”.

There are a number functions availble in PHP to do this, each doing it in a slightly different way:

You are reading a book sample. This rest of this section is available in the full book.

6.2 Talking to other processes

Once we’ve got other processes up and running, PHP supports a number of ways to talk to them, which we’ll look at below. These methods mainly apply to processes that have been started up by other means, or which your script has started and disassociated itself from. If you’ve used popen() or proc_open() to run them, you can simply communicate via PHP streams as normal. You can still use the methods below as well though in addition, if you need additional channels of communication.

You are reading a book sample. This rest of this section is available in the full book.

6.3 Semaphores

You are reading a book sample. This rest of this section is available in the full book.

6.4 Shared Memory

“Shared Memory” is a simple and widely supported method of passing data between two or more processes, and it really is as simple as it sounds. A process can create a segment of memory, with a key as a unique ID, assign standard Unix type permissions to it, and then it and other processes can read, write and delete data from that segment as necessary (and as permissions allow).

Shared memory is probably the fastest way of sharing information between two processes. There is no disk IO or database access for instance to slow you down, and no intermediate message broker to spend time processing and distributing the information. The downside is that you have to manage the whole process yourself, although this isn’t as hard as it sounds.

Shared memory is supported in two extensions, the SystemV sysvshm extension (part of the Semaphore extension like sysvsem described in the previous section) and the more recent shmop extension. We will look at the latter as it is based on the C shm api which makes it easier to share data with non-PHP programs, unlike the former which used its own proprietary data format making it hard to share with anything other than PHP. Shmop is also usually faster than the sysvshm as it stores its data in a raw form.

Like files, shared memory can become corrupt if multiple processes try to write to it at the same time. For our example below, we don’t need to worry about locking the shared memory segment as only one process will be writing to it so we won’t include any locking code for brevity and clarity. However where locking is an issue, you can use the semaphore method outlined in the previous section to ensure only one process writes to the memory segment at a time. Using multiple memory segments at the same time by the same or different processes doesn’t require locking, as long as only one process is writing to a specific segment at a given time.

It’s time to look at an example of shared memory in action. Below are two scripts. The first script below, generator.php, generates an array of three random numbers every second, encodes them as JSON and puts them into a shared memory segment. The second script, display.php, retrieves that data and outputs it to the terminal. To try these, open two terminal windows and run one in each, at the same time.

You are reading a book sample. This rest of this section is available in the full book.

6.5 PHP message queues

Message queues provide an easy way for multiple processes to interact. At its most simple, one process adds messages to the queue, and another takes them off. The queue is maintained usually by a intermediate process or the OS itself. Queues make a programmers life easier than, say, messaging using shared memory as the programmer doesn’t need to worry (too much) about timing and synchronisation, and there are usually no need for locks. The sending process can fire off a message and forget about it (if its wants to), the receiving process can simply sit around and wait for a message to arrive (or do other things and check back for the messages later). There are a number of message queue APIs and extensions available to PHP, and we will look at most of those later in this chapter. In this section, we will look at the sysvmsg message queue which is part of the Semaphore extension that we looked at in the previous sections. This is a basic but useful message queue system, and has the advantage that it is usually compiled into most PHP distributions and it doesn’t require any third party APIs, daemons or libraries.

Lets jump right in with an example. We’re going to modify our random number generator scripts from the previous section to use a message queue. Our first script, generator2.php is going to fire off 20 messages, each with 3 random numbers, in quick succession and then exit. Our second script, display2.php, will pull those messages out at its own pace (1 per second) and display them.

You are reading a book sample. This rest of this section is available in the full book.

6.6 Third party message queues

You are reading a book sample. This rest of this section is available in the full book.

6.7 APC cached variables

You are reading a book sample. This rest of this section is available in the full book.

6.8 Virtual files - tmpfs

You are reading a book sample. This rest of this section is available in the full book.

6.9 Standard streams

You are reading a book sample. This rest of this section is available in the full book.

6.10 Linux signals

You are reading a book sample. This rest of this section is available in the full book.

6.11 Task dispatch & management systems

You are reading a book sample. This rest of this section is available in the full book.

7 Talking to the system

So far we’ve looked at software that communicates with your users, via text based or graphical interfaces, and system software that doesn’t need to talk to users at all. One thing that both types of software have in common is the need to deal with the underlying system that it sits on top of. That system is a structure containing the file system, operating system, hardware interfaces and various system level services.

When programming for the web you typically don’t interact with hardware, lower level aspects of the system and so on. Indeed in many cases you specifically take steps to prohibit your users from doing so! In contrast, dealing with printers, sound-cards and other hardware is a common requirement when constructing many types of software, and interfacing with system level services is often a necessity. You will work with portions of the file system from web pages, but with off-line software you usually have the freedom and resources to work with a wider range of file formats, larger file sizes and more privileged files.

In this chapter we’ll look at some of the different ways to interact with these resources both from within PHP and with the aid of helper applications, and some of the issues to consider when doing so.

7.1 Filesystem interactions

There are many different types of data files that software needs to commonly interact with, from images to text files, formatted documents to videos, structured data to configuration files, and many more. PHP has built in functions for reading, writing, parsing and displaying many different types of data files, and between Pear, Pecl, Composer and third party libraries even more types are covered. On many systems, particularly unix variants, helper applications can also be called to further extend the range of file types covered. In fact, there are very few file types that you will struggle to deal with in PHP, and those tend to be proprietary formats with closely guarded specifications. If you stick to open formats, and in particular standards-based formats, you will invariably find the tools you need in the PHP eco-system.

7.2 Data files & formats

Appendix B contains a reference list of functions/libraries/helpers available for a wide range of common formats. Remember that where a particular version of a format doesn’t appear, it is often usable using the functions for the generic format it is based on (e.g. many XML based formats are perfectly at home being manipulated by the XML tools listed).

Always bear in mind that data files are a large vector for security exploits, and even where software is operated locally by trusted users, those users may inadvertently try to open files from malicious sources. Always treat external data as potentially tainted, and treat un-vetted extensions/helpers as if they have potential security vulnerabilities.

7.3 Dealing with large files

When you don’t have a limit on the time your script can run, you will find that you can work with bigger files than you may be used to when using PHP on the web. In general, you can deal with them in the same way as you would with smaller files. However one big problem you may run into is memory usage. It’s important to understand how PHP uses memory when loading and processing files so that you can make appropriate choices in your code. Many of the libraries for handling particular file formats listed in Appendix B will deal with opening and processing large files efficiently on your behalf, so this section is most relevant when you do your own file processing, or use a library that requires you to pass it raw data (rather than a filename).

First lets look at simple methods for reading in a file in one go. PHP has two easy to use functions for doing this, file() and file_get_contents(). The former reads the file into an array, the latter into a single string.

 1 <?
 2 
 3 $filename = 'bigfile.csv';
 4 
 5 echo("Size of file : ".filesize($filename)." bytes\n");
 6 
 7 $memory1 = memory_get_usage();
 8 
 9 $file_array = file($filename);
10 
11 $memory2 = memory_get_usage();
12 
13 $file_string = file_get_contents($filename);
14 
15 $memory3  = memory_get_usage();
16 
17 echo("Memory used by array : ".($memory2-$memory1)." bytes\n");
18 
19 echo("Memory used by string : ".($memory3-$memory2)." bytes\n");	

Running this on a sample large file I had lying around, gave the following output

1 Size of file : 186097433 bytes
2 Memory used by array : 296969824 bytes
3 Memory used by string : 186097588 bytes

So we can see that reading the file (about 177Mb) into a string adds an overhead of 155 bytes, which is not too bad at all. However reading this file into an array adds an additional 105Mb to the original size of the file! In PHP, arrays are a very versatile data structure, they “can be treated as an array, list (vector), hash table (an implementation of a map), dictionary, collection, stack, queue, and probably more” (according to the PHP manual). However this versatility comes at a price, and that is the additional memory used for storing the structure information. So before loading the file, consider whether the processing you are going to do on the data can be done on a string, or whether the extra overhead of an array is worth it for the manipulation capabilities. If you need a traditional array or hash like structure without the versatility of the PHP array type, look into the PHP SPL (Standard PHP Library) which contains a range of “traditional” data structures that may be more optimised for your use case.

Further Reading

SPL book in the PHP Manual
http://www.php.net/manual/en/book.spl.php

Sometimes, no matter what type of data structure you read your file into, there isn’t enough memory available on the system, you hit a memory limit imposed for your script, or you just need to keep memory usage low in general. Often processing of a datafile can be done on a line-by-line (or chunk-by-chunk) basis, and PHP allows us to read in a file piece by piece rather that in one go. Assuming you discard the data you’ve read before you read some more (unset it, overwrite it or write it out to a file, for example), then you will just use enough memory to store that one line or chunk.

 1 <?
 2 
 3 $filename = 'bigfile.csv';
 4 
 5 $memory1 = memory_get_usage();
 6 
 7 $file_string = file_get_contents($filename);
 8 
 9 $memory2  = memory_get_usage();
10 
11 unset($file_string);
12 
13 $memoryBase = memory_get_usage();
14 
15 $file_handle = fopen($filename, 'r');
16 
17 while ($line = fgets($file_handle)) {
18 
19 $memoryCurrent = memory_get_usage();
20 
21 if ($memoryCurrent > $memoryBase) { $memoryHigh = $memoryCurrent;};
22 
23 };
24 
25 echo("Memory used by single string : ".($memory2-$memory1)." bytes\n");
26 echo("Max memory used when reading by line : ".
27 		($memoryHigh-$memoryBase)." bytes\n");

On my sample file this gave the output :

1 Memory used by single string : 186097768 bytes
2 Max memory used when reading by line : 9000 bytes

which should illustrate the extreme differences in memory usage using the two different techniques.

If you’re working with files that are, or may be, greater than 2Gb in size you should also be aware that some filesystem functions may not return the correct (or any) result for files bigger than that on many platforms. This is because these platforms use a 32-bit integer, PHP’s integer type is signed, and 2Gb is the largest size that can be represented by a signed 32-bit integer. This affects functions like filesize(), stat() and fseek(). You can of course access external commands to replace some of these functions, for instance wc -c on Linux will return the number of bytes in a file for all files supported by the operating system. On 64 bit Linux, with a recent version of glibc installed, you can compile PHP with the D_LARGEFILE_SOURCE -D_FILE_OFFSET_BITS=64 flag for better large file support, though be aware if you are writing scripts for distribution that this obviously won’t be an available option for all.

In all of these examples, remember that your script has to work within any memory limit you or PHP has imposed. By default the PHP CLI SAPI turns off the memory limit, you can type php -r "echo(ini_get('memory_limit'));" at the command line to see the default memory limit (-1 means no limit). From within your script, ini_get('memory_limit') will tell you the current maximum, and memory_get_usage() will tell you what you’re currently using.

Handling memory usage when dealing with large amounts of data is something that often trips programmers up. As we’ve seen, understanding how the various functions we use operate can help us process that data more efficiently and help us to minimise the memory we use.

7.4 Understanding filesystem interactions

PHP has a number of filesystem related extensions, many of which are part of the PHP core or compiled into most PHP distributions and contain hundreds of useful functions. Most of them operate in a simple straight-forward manner, allowing you to get and set file and directory information and manipulate files and the filesystem as you would expect. We won’t cover most of them here, as I’m sure many will be familiar to you from your web projects and they are covered well in the PHP manual.

Further Reading

Filesystem related functions in the PHP manual
http://www.php.net/manual/en/refs.fileprocess.file.php

Many of these functions ape command line programs that you may be used to, like chmod, mkdir, touch and so on, and operate broadly how you expect. However there is a difference that raises its head in longer running script, and revolves around PHP caching filesystem information to increase performance, which we will look at next.

7.5 The PHP file status and realpath caches

On the web, speed is king. PHP operates two information caches to speed up access to the filesystem. The first is the file status cache, which caches information about a given file (like whether it exists, whether it is readable, it’s size and type, and so on). The second is the realpath cache, which caches the actual, real path for a given file or directory (expanding out symlinks, relative paths, ‘.’ and ‘..’ paths, include_path’s and so on). Information is added to the cache automatically by PHP each time it encounters a new file, and is then used by any number of functions the next time they attempt to look at that same file. With a web page that’s gone in the blink of an eye where little may have happened on the filesystem, this is often a good trade-off for increased performance.

However the chances that the details of a file or path may change while your script runs obviously increase with the length of time that your script takes to execute. Therefore PHP gives us a couple of options for working with these two caches.

The following example shows the file status cache in action, and how to use clearstatcache() to clear it.

You are reading a book sample. This rest of this section is available in the full book.

7.6 Working with cross platform filesystems

You are reading a book sample. This rest of this section is available in the full book.

7.7 Accessing the Windows Registry

The Windows Registry is a structured hierarchical database which Windows and other applications use to store configuration information. While not used universally by all applications, most will store some information in the Registry, and the operating system itself uses it extensively. We can access the registry from PHP too, allowing us to check and set configuration information both for our own applications, and if we have the right permissions, other applications and the OS itself. To do this, we need to use the win32std extension with PHP.

You are reading a book sample. This rest of this section is available in the full book.

7.8 Linux signals

On Unix and Linux systems, “signals” are a method that the Operating System (possibly at the behest of a user) can use to send, er, signals to a process. The OS interrupts the normal execution flow of the process to deliver the signal, allowing the process to act on the signal immediately (if it wishes to do so). There are a wide range of signals that can be passed, including requests to terminate, error condition signalling and polling notifications. Common signals have been codified in the POSIX standards and a full list can be found on Wikipedia, and the list supported by PHP can be found in the PHP manual :

Further Reading

POSIX signals on Wikipedia
http://en.wikipedia.org/wiki/Unix_signal#POSIX_signals

PHP supported signals in the PHP manual
http://www.php.net/manual/en/pcntl.constants.php

We can listen and respond to these signals from our PHP scripts using the PCNTL extension. The following script demonstrates how to do this. It uses a PHP feature called “ticks”, which allow a callback function be executed after every N statements. We don’t need to do anything with ticks, we simply need to enable it (using a declare construct) so that it is available to the PCNTL functions to use as they deem fit.

You are reading a book sample. This rest of this section is available in the full book.

7.8.1 Sending Signals

You are reading a book sample. This rest of this section is available in the full book.

7.9 Linux timed-event signals

Sometimes we want our scripts to do something every-so-often, for example checking the status of a resource, do some clean-up, update a log file or similar. There are a couple of ways to achieve this. The simplest is by using the sleep() or usleep() functions to sit and wait for a number of seconds/microseconds before performing a task. This is not always of use, as when you call sleep() the script simply stops and waits for that amount of time rather than continuing to do other useful work. In the previous section we briefly looked at PHP “ticks”, which allows us to run a callback function every N (potentially useful) statements. This allows us to do useful work in-between calls to the callback function, however there is no guarantee on how long those statements will take to execute so we can’t wait for a specific length of time. In fact, PHP has no internal way of keeping track of time in this way. However using POSIX signals, which we looked at in the previous section, we can ask the system to set an “alarm” for us a certain number of seconds into the future. When the “alarm” goes off, the system will interrupt PHP with a signal, which we can handle to run our callback function.

You are reading a book sample. This rest of this section is available in the full book.

7.10 Printing (to paper)

You are reading a book sample. This rest of this section is available in the full book.

7.11 Audio

You are reading a book sample. This rest of this section is available in the full book.

7.12 Databases - no change here

You are reading a book sample. This rest of this section is available in the full book.

7.13 Other hardware and system interactions

You are reading a book sample. This rest of this section is available in the full book.

8 Performance & stability - profiling and improving

Or, “Why is PHP sooooo slow” (hint - it isn’t, mostly)

PHP script performance (a term which we use to encompass indicators such as speed of execution and usage of resources) is an issue for both PHP based web sites as well as other applications written in PHP. This chapter looks at the issues affecting PHP performance in general, specific performance considerations for non-web applications, and the tools and resources available for solving performance related problems. We also look at stability of long running scripts, which is often tied closely to performance issues.

8.1 The background on performance

PHP is generally considered a “scripting” or “interpreted” language. This means that rather than compiling or transforming source code into machine executable instructions and distributing those as a standalone program (as is usually the case with languages like C), PHP programs are distributed as PHP source code. The user of the software requires the PHP interpreter (also known as the PHP Virtual Machine) to run that source code, with the interpreter converting the PHP source code to machine instructions on the fly as the application executes. This style of execution has upsides and downsides. The main upsides are ease of code updating and deployment (no compilation steps) and fewer architecture issues (write-once-run-anywhere, no need to compile for specific platforms). The main downside is the performance hit of the interpretation stage as the application runs. Modern versions of PHP actually reduce this performance hit by first transforming the code into intermediate “opcodes” which are then executed. Those opcodes don’t need to be re-interpreted every time the same code occurs (e.g. in a loop). For frequently used scripts, such as web scripts and some applications, these opcodes can even be cached between script runs for faster subsequent execution.

PHP is also a “high-level” language, which means that it uses abstractions to hide much of the lower-level detail of the code the computer needs from the user. Instead of dealing with implementation details like memory addressing and call stacks, PHP and other higher level languages present the computer to the programmer with abstract concepts such as variables, arrays and functions. Much of the appeal of a language like PHP is the wide range of in-built functions and operators to perform common tasks, which leads to higher developer productivity. Versatile data structures (like PHP’s array type, which is actually a type of managed ordered map that can be used as a traditional array, a list, a hash table, a dictionary or several other common data structure types) hide details of memory management, simplify access, and provide a rich variety of related manipulation functions. However these abstractions comes at a performance cost. Behind every function, array operation, or disk access are lower level algorithms written in C. The implementation of these algorithms must cater to the “general” case and include all possible uses of the algorithm. Thus they will rarely be as optimal as C algorithms written specifically for your particular need. This level of abstraction again adds an overhead to PHP when compared to lower-level languages like C, although the core PHP developers have invested a lot of time and effort in the last couple of years to optimise, trim, re-factor and otherwise increase the performance of the C back-end to great effect, wildly increasing speed and reducing memory consumption in many common cases. And for those common cases, these pre-built PHP algorithms are often better than those you may be able to write yourself!

The final reason for performance problems is perhaps one of the most prevalent issues but also one of the easiest to fix. And that reason is the PHP programmer. Many developers aren’t aware of how to find and solve performance issues in their own code, often blaming their own failings on PHP itself even where PHP is just as performant as other languages. While PHP takes away a lot of the pain and slog from programming, hiding and dealing with numerous tasks for you, it is still a general purpose programming language and as such it is still perfectly possible to write poorly performing code. Even with an appreciation of the issues, many developers aren’t aware of the tools and resources that are available to help them improve their programs performance.

For those that doubt that it is possible to write high performance systems in PHP, the set of slides in the Further Reading section below provides examples from one company that shows that it’s not just possible, it happens in the real world. Appendix D also shows some of the things that people are doing with PHP, where performance clearly isn’t holding them back.

Further Reading

“More Than Websites:PHP And The Firehose @ Datasift” by Stuart Herbert
http://blog.stuartherbert.com/php/2013/04/16/slides-from-brighton-php-more-than-websites-php-and-the-firehose/

8.2 Specific issues for general purpose programming

As noted above, both web and non-web PHP scripts can have performance problems. However there are some additional performance related issues to take into account when programming longer running scripts and those without memory restrictions. In the CLI SAPI, as with the traditional web PHP model of programming, PHP manages limits on memory consumption and executes garbage collection on your behalf. This works well on the web where the shorter lives & lower resource intensiveness of many scripts mean limits and management processes are rarely noticed. However when you program a general purpose application you will often want to remove the imposed memory limits to allow your application to consume all the memory it needs on different systems with different amounts of memory. Indeed, the default configuration for the CLI SAPI now sets the memory and execution time limits to 0, unless you specify otherwise. This transfers the burden of managing and limiting memory usage to your script. Likewise, in longer running, resource intensive and response-time-sensitive scripts garbage collection requires a different approach to avoid unwanted blocking or unnecessary conservancy. This involves the manual management of the garbage collection process within your scripts.

Inefficient programming and algorithm design on your part as the programmer can also be more noticeable than it is on the web. The time that an algorithm takes when generating a web page can get lost in the other time overheads of transmitting and displaying a page, but when the exact same algorithm is executed on the command line a noticeable pause may be visible. Responsiveness to a user is critical on the web, but also very noticeable to local users as well.

The rest of this chapter looks at how to profile and manage the performance of your scripts and the resources they use, including strategies to target the problems described above.

8.3 Profile, profile, profile!

We’ve all come across slow running scripts (often our own!), and usually the first response is to start looking up ways to increase PHP’s speed. Compiling, caching, re-factoring code, accelerators; these are topics that googling for PHP performance or speed issues will readily turn up, we may have already read about them, and then we dive right in to try them out. My advice, (derived from bitter personal experience), is to STOP RIGHT NOW. Throwing performance trick after performance trick against your code (often that you find online or in good books like these), even where they appear sensible and you can see the logic, can end up complicating your code or adding additional dependencies for no good reason. Why? because when we don’t know what the root cause of the problem is, we don’t know if a particular solution, no matter how good on paper, will actually addresses the issue we are having in this particular case. And even if it does appear to work, we don’t know if it was the simplest way to fix it, and thus whether we’re saddling ourselves with extra “technical debt” when we don’t have to.

The step we often miss out is to ask our script directly : “why are you running so slowly?”. If our script tells us we can then attempt to fix the issue directly without the use of external tools like compilers and caching systems. So how do we ask our script the “why” question? By “profiling” it.

A profiler, if you’re not familiar with the term, watches a piece of software (usually from the “inside”) as it runs and breaks down the time (and sometimes resources) that each part of the program uses. The profile information is often reported down to the level of an individual line of code or function call. This helps us spot exactly where our scripts are slowing down. Is it that complex database query? A badly written loop? A function that’s called more times than expected? Disk or network access pausing execution? What ever the problem, the profiler will tell us. Once we know exactly what the cause of our slowdown is, the solution to the problem is usually apparent (or at the very least we can rule out potential solutions that won’t actually fix it). It may just mean re-writing a few lines of code or caching some data instead of repeatedly generating it. It may point out problems external to PHP, such as a slow database server, or laggy network connection or resource. Of course in some cases it may end up being an intractable problem from a PHP programming point of view that does indeed require the help of an accelerator or external caching system. In any case, we will likely save time and prevent making unnecessary changes to our code or deployment environment by using a profiler to ask the “why” before we start trying the “what”.

With PHP you have several choices when it comes to profiling. You can manually profile your code by adding profiling/measuring statements directly to your code base, or you can use one of a number of tools to automatically profile your code for you. The former is quick and easy to do with no changes to your development environment, if you know roughly where in your code the problem lies. The latter, whilst requiring the setup and configuration of the tools, and learning how to use them the first time, provides more comprehensive profiling. It also doesn’t rely on you knowing where your problems may be located, and usually requires minimal or no changes to your code base. We’ll look at both options below.

8.4 Manual profiling

Manual profiling entails adding additional code to your source to measure time or resources directly from within the scripts. The following is an example of measuring execution time of different lines of code.

You are reading a book sample. This rest of this section is available in the full book.

8.5 Profiling tools

You are reading a book sample. This rest of this section is available in the full book.

8.6 Low level profiling

You are reading a book sample. This rest of this section is available in the full book.

8.7 Profiling - the likely results

Profiling can reveal an inordinate number of different performance problems. However, the following are some of most common types of problems discovered when profiling PHP, along with some strategies for addressing them :

You are reading a book sample. This rest of this section is available in the full book.

8.8 Silver bullets

Silver bullets are “solutions” that you can “throw” at your script, which will hopefully speed them up without you having to think too much about the cause of the slowdown. In the first version of this section that I wrote, the words “silver bullet” were always written with a question mark after them. I took it out as it cluttered the place up, but the point was that there are no guaranteed, universal silver bullets for increasing performance in PHP. Each of the oft-called “performance solutions” below have downsides, aren’t suitable for everyone, and require a reasonable amount of thought to implement. Nevertheless they can be useful, particularly when you have improved your script as much as you can manually and you are stretching the capabilities of PHP or the available hardware.

8.9 Silver bullet #1 - Better hardware

If funding allows, sending out for a beefier server or desktop can often produce instant performance gratification. In some cases hardware is cheaper than developer time, so it’s a no-brainer. However, there are some downsides, particularly for those who are cash-challenged:

You are reading a book sample. This rest of this section is available in the full book.

8.10 Silver bullet #2 - Newer PHP versions

You are reading a book sample. This rest of this section is available in the full book.

8.11 Silver bullet #3 - Opcode caching

You are reading a book sample. This rest of this section is available in the full book.

8.12 Silver bullet #4 - Compiling

You are reading a book sample. This rest of this section is available in the full book.

8.13 Silver bullet #5 - JIT compilers and alternative Virtual Machines

You are reading a book sample. This rest of this section is available in the full book.

8.14 The SPL - Standard PHP Library

You are reading a book sample. This rest of this section is available in the full book.

8.15 Garbage collection

You are reading a book sample. This rest of this section is available in the full book.

8.16 Multi-threading and concurrent programming in PHP

You are reading a book sample. This rest of this section is available in the full book.

8.17 Big data and PHP - MapReduce

You are reading a book sample. This rest of this section is available in the full book.

8.18 Data caching

You are reading a book sample. This rest of this section is available in the full book.

8.19 Know thy functions

You are reading a book sample. This rest of this section is available in the full book.

8.20 Outsourcing code to other languages

You are reading a book sample. This rest of this section is available in the full book.

8.21 Other performance tips and tricks

You are reading a book sample. This rest of this section is available in the full book.

8.22 Stability and performance of long running processes

It’s not just performance that can be affected by the issues raised in this chapter. Stability of your scripts is also affected by the poor management of resources and non-optimal algorithms. By stability, we mean, essentially, “crashes”. PHP was initially “designed to die”, that is, it was designed for shorter running scripts where the prospect of still being running many minutes later was not a big issue, let alone many hours or days later. That said, if you program carefully there is no reason why you can’t create scripts (usually daemons) that can run pretty much indefinitely.

The key areas to think about when designing a stable program are :

You are reading a book sample. This rest of this section is available in the full book.

8.23 Avoid micro and premature optimisations

Now that I’ve convinced you of the need to consider performance and instructed you on the finer arts of speed and resource management in PHP, I’m going to try and convince you to put it to the back of your mind, at least for now. It is easy to get caught up with performance and optimisation and let it affect your workflow and productivity. While it is generally positive to follow good practice and consider the performance of your code at all stages of development, premature optimisation is often just that - premature, and micro-optimisations are not usually worth the paper they are written on.

As your author I have an embarrassing confession to make. Having started programming in PHP many years ago, when the performance difference between using single and double quotes around a string was measurable in some contexts (or at least it was “common knowledge” that it was), I still find myself automatically considering what type of quote to use each time I type out a string, not for syntactical reasons but for performance reasons. That’s despite the fact that the performance difference between the two now-a-days is definitely extremely, extremely negligible. Old habits die hard though, and many of the micro-optimisations that you will find on the web fall into a similar category. Profile your code, and you will quite likely find that a single verbose database call will make the time saved by calling isset() dozens of time to check variables instead of falling back on @ to simply suppress warnings look incredibly small (that’s not to say that there aren’t other good reasons to do that though!). Developer time in writing and thinking about micro-optimisations is typically much more expensive than the extra processor cycles used to execute un-optimised code.

Premature optimisation typically occurs when developers start thinking about how to scale their code, before they’ve got any of the code working and deployed. There is no point working out how to scale code that may never see the light of day because the project over-runs and isn’t delivered. Unless you are sure of the volume of customers/users/data you will be processing, its usually a better idea to get something up and running, as a proof of concept if nothing else, and then scale it when you know that it will be needed. Indeed, scaling methods vary considerably depending on exactly what your code needs to do, and often this is not finalised until you have something up and running. So even if you deliver on time, your scaling efforts may all be in the wrong direction compared to how your project ends up functioning.

“But, but, but…” you cry, your project is different. It may well be, you might be lucky enough to work on a large scale well funded project that has a known high performance requirement, or be developing a tool that is specifically geared towards high performance uses. Congratulations, you can ignore this section (if you’re even at the level where reading this book is useful!). For the rest of us, and this is the majority of PHP projects, concentrate on getting it working and deployed first. Keep performance in mind during development, but don’t let it interfere with your work flow too much. Most projects fail for reasons other than poor performance.

Further Reading

“PHP: Require/Include vs Autoloader”. An example of an article that espouses an optimisation method that is definitely premature (if worth it ever). The article shows that you can save a whole 6 milliseconds in total, in a script that runs the functions 100 times. That’s not even worth thinking about for most people, who would only call it a hand-full of times at most, and in any case for whom 6 milliseconds isn’t a concern in the slightest.
http://www.endyourif.com/php-requireinclude-vs-autoloader/

You are reading a book sample. This rest of this section is available in the full book.

9 Distribution and deployment issues

Now you’ve written your perfect peice of software, your greatful audience awaits it’s delivery and installtion onto their eager machines. Distributing and deploying software comes with its own set of headaches above and beyond those you encountered just writing the damn thing. this chapter covers some of the issues that you may come across when thinking about deployment and distribution of your software, that may vary from the common situation of pushing code to your own server for a PHP website. In particular, we are looking at scenarios where you distribute your software and it is no longer under your direct control and not on your own systems.

9.1 Error handling and logging

You will need to think about how you handle and log errors (and other information) in your scripts when you run with the CLI SAPI. You can use the normal PHP error handling functions and allow PHP to log errors to the file specified in error_log in php.ini. However bear in mind that you may not be able to control which user your software is run under, and so you need to log to an appropriate location where you can be sure your user has appropriate permissions to write to.

One alternative to the standard “log to a file” method in PHP is to instead send errors and other log information to the system’s syslog. In Linux systems this is syslog(3) (man syslog gives more information), in Windows this is the system event log. Using syslog has several advantages :

You are reading a book sample. This rest of this section is available in the full book.

9.2 Installers and bundling files

In Chapter 3 we looked at different ways to run PHP, but before you can run your PHP scripts you need to get them (and any related assets) onto your target machine. Generic software installer systems are widely available and many platforms have their own software repositories and bundling systems which are well documented, so we won’t cover those in any detail here (although we’ll mention a few good open source ones at the end). Instead, we’ll have a look at a couple of PHP specific options for bundling files and resources ready for distribution and installation.

9.3 Embedded data files at end of PHP script

If we just need to include a chunk of data with a PHP script, we can use a language construct introduced in PHP 5.1, the __halt_compiler() instruction. Simply put this at the end of your PHP script, and you can then put anything you want (text strings, binary data, another PHP script etc.) following it. When PHP execution hits the __halt_compiler() line, it simply stops. This means that whatever follows isn’t executed, and won’t throw any syntax errors and so on. Your script can access the data that follows it by opening a file handle to itself and seeking to the __COMPILER_HALT_OFFSET__ constant which is created when a __halt_compiler() instruction is present. The following example shows the “unbundling” of a text file from a PHP file.

 1 <?
 2 
 3 # Open a pointer to this file using the magic constant __FILE__
 4 
 5 $thisfile = fopen(__FILE__, 'r');
 6 
 7 # Seek down to the 1st byte after the __halt_compiler(); instruction. This
 8 # is contained in the automatically created __COMPILER_HALT_OFFSET__ 
 9 # constant.
10 
11 fseek($thisfile, __COMPILER_HALT_OFFSET__);
12 
13 # Lets grab everything that follows that ...
14 
15 $ourtext = stream_get_contents($thisfile);
16 
17 # and write it out to a new file.
18 
19 file_put_contents('textfile.txt', $ourtext);
20 
21 # Our scripts stops here. 
22 
23 __halt_compiler();The additional content starts here.
24 
25 This is the text file.
26 It would normally cause a PHP fatal syntax error if 
27 this text was simply dumped into a PHP file.

If you run this script, and then look at the created textfile.txt file, you will see that it contains the rest of our script starting with the The additional content starts here line.

Although we’ve used plain text here, as it’s easier to show in a book, there is nothing to stop you adding binary files, PHP files, or even tar/zip archives containing multiple files which your script can save and expand. However if you are intent on including multiple files or large amounts of data, you might want to consider the Phar format option, discussed next.

9.4 Phar executable bundles

Phar bundles are a native PHP way of pulling together lots of files, both PHP code and ancillary data files, into a single file for distribution. Phar bundles are, at their core, either a zip file, tar file or custom phar format file. You can access the individual files in the bundle using the phar stream wrapper as you would a normal file, without having to unbundle the files before use. For example, if you have a bundle called mybundle.phar, you can do :

You are reading a book sample. This rest of this section is available in the full book.

9.5 Generic installers

You are reading a book sample. This rest of this section is available in the full book.

9.6 Controlling the (PHP) environment

When you deploy scripts on your own servers or PCs, you can control the environment in which they are deployed. However if you are distributing your software more widely you will suddenly find that environmental factors on external machines can make your PHP scripts misbehave or even stop working completely. Some of the PHP specific environmental factors you need to consider include:

You are reading a book sample. This rest of this section is available in the full book.

9.7 Extending your application with plug-ins

You are reading a book sample. This rest of this section is available in the full book.

9.8 Documentation

You are reading a book sample. This rest of this section is available in the full book.

Warning

Note: All the legal information presented below and elsewhere in this book may be incorrect, depends on your jurisdiction, and was believed correct by the non-legally-trained author only at the time of writing. The information is only provided to give you an overview of the topics that you may wish to consider in respect of your particular project. Always consult a qualified legal professional for legal advice. Never drink coffee just before bed.

If you’re coming from a web development environment to, for instance, desktop software development, it is worth taking a moment to think about the consequences and differences in licensing and legal liabilities in your new mode of programming, particularly when programming commercially. With the web, your “product” is usually your website content, not the PHP script itself. Selling software reverses the situation, with your carefully honed PHP code being the saleable commodity and the output belonging to the end users.

You are reading a book sample. This rest of this section is available in the full book.

9.10 Deploying frameworks

You are reading a book sample. This rest of this section is available in the full book.

10 Where now? or, Thanks & feedback

If you’ve read this far, thank you. I sincerely hope that this book has held your interest and at the very least informed some areas of your future PHP programming. If it has, I would encourage you to start some coding right now based on one or more topics in this book while they are still fresh in your mind. People far smarter than me have shown that the sooner relevant activity occurs after learning, the easier it is to retain information and techniques over the longer term. If you use any of the techniques presented in the book in “real-life”, I would be very interested to know! Finally, if you’ve not already headed for the keyboard, don’t forget to glance through the Appendices that follow. There may be something interesting or useful in there for you (either to us now or to be aware of for the future).

10.1 Giving feedback, and getting help and support

E-mail : author@phpbeyondtheweb.com

Your feedback on this book, good or bad, fundamental or trivial, is solicited and very much welcomed. Tell me what you thought about the book, overall or a particular section. Let me know if any areas weren’t covered in enough depth (or in too much detail), if any topics you were expecting weren’t present, or any of the information wasn’t clear. I intend to keep revising and improving this book over time, as well as adding new topics and example code, and as an existing purchaser you will get free updates as newer versions are produced. Likewise, if you have any problems getting the sample code to run, or issues implementing the techniques discussed please do drop me a line and I’ll see if there is any way I can help.

10.2 Giving testimonials

If you would like to provide a short testimonial for this book’s website, please e-mail it to author@phpbeyondtheweb.com indicating you are happy for me to use it to promote the book. In return, if I use it I will provide a link back to your website/twitter/email.

10.3 Are you reading a “pirated” copy?

There are times in my life when I have read “unauthorised” copies of books; when I couldn’t afford them, when they weren’t easily available through legitimate channels (local bookshops, online sales, libraries), or (for e-books) when DRM thwarted my efforts to read them on the devices I had available. So I quite understand if you are in this position, and I don’t want to criticise your choice. If you are unable to afford this book, perhaps you could consider purchasing it in the future when you are able, and in the meantime I would ask that you do me a favour in return - please consider helping me to promote it by recommending it to those that can afford it, have a go at reviewing it online, and otherwise spread the word. If you’re reading an unauthorised copy because there was something else (other than cost) blocking your access to a legitimate copy, please let me know and I’ll do my very best to remove that obstacle. I want to help everybody who wants to buy and read this book to be able to do so with the minimum of fuss. This is why I won’t employ DRM on my work, because for every “pirate” that it stops (and it won’t be many), there are likely to be ten times as many legitimate readers who it annoys.

If you are reading an unauthorised copy because you simply don’t want to pay for it, then that is between you and your conscience. That is how you have chosen to lead your life, and I have nothing more to say to you.

If you are selling or otherwise making money from knowingly distributing unauthorised copies of this book, then I will do my level (legal) best to stop you. That’s just not nice.

Appendix A : Compiling and installing PHP and its extensions

There are a dozen different ways to get PHP, including downloading & compiling it yourself, downloading pre-compiled binaries, using package managers and software repositories, and finding it pre-installed by a forward thinking administrator. On most Linux distributions, PHP can be installed with a one-line command such as sudo apt-get install php5 or through graphical package managers like the Synaptic Package Manager or the Ubuntu Software Centre. Many common PHP extensions and add-ons are likewise available as pre-built packages or alternatively through the PECL and PEAR systems.

However there are times when it becomes necessary to do a little more work to install PHP, such as :

  • when your project has requirements for a specific version of PHP that is different to the one shipped with your OS
  • where you need extensions not available as packages,
  • or when you want to compile a customised version of PHP specific to your needs.

Like anything involved in computers and software development, compiling PHP can take you down a rabbit-hole of options, customisations, compatibility issues, libraries and dependencies. A whole book could be written about the different possibilities (and possible headaches) involved. Luckily for us, in most use cases and needs, the basics of compiling a standard version are quite straightforward. And like most things in life, it gets easier once you have done it once. The first section below will go over the steps necessary for getting, compiling and installing PHP and it’s core extensions. PHP is written in C, and as many of you may not be familiar with the process of compiling C programs, I have tried to explain each step below to give you an idea of what is happening. This makes the process seem a little more verbose, but in reality it is quite straightforward. Go ahead and try it if you don’t believe me! The next sections are also worth a read, as they cover installing extras like libraries and extensions from the PECL, PEAR and Packagist repositories.

Compiling and installing PHP itself

Note for Windows devs - These steps are for Linux/Unix type systems, and use free compiler tools almost always included with the OS. For Windows, the proprietary Visual Studio compiler is required, and the steps are somewhat different and beyond the scope of this book. Windows source code, pre-built binaries and instructions for compiling can be found at http://windows.php.net/download/ , with older versions in the archive at http://windows.php.net/downloads/releases/archives/.

Our first step is to download the PHP source code from the PHP website at http://www.php.net/downloads.php . This page lists the current stable release and the previous supported stable release. Newer versions which are still under development are available at http://snaps.php.net/, and older end-of-life versions are available at http://museum.php.net/ . Git users can also pull the source code down from the official mirror at https://github.com/php.

When you have identified which version you want, make a note of the URL of the .tar.gz source code file which we will use later.

You are reading a book sample. This rest of this section is available in the full book.

Compiling and installing (extra) core extensions

As we saw in the previous section, the most common way to install core extensions is to enable the relevant flags at the configure stage during compilation of the main PHP installation (note by default, many extensions are automatically enabled). However it’s not uncommon to come across the need to install an additional extension later on, for instance as the requirements for your program change from its initial design. There are two ways to approach this. The first, which you’ll find recommended a lot online, is to re-do the compilation/installation of PHP from scratch, adding the newly required modules at the configure stage (after issuing php -i to remember what configure options you used the first time). While this works perfectly well, compiling the full PHP binaries is a bit of a slog which can take older PCs in particular a long time to complete. There is a short cut however.

Each of the core extensions is actual a separate .so binary, and can be compiled independently. To do this, follow the first steps in the previous section to download and unpack the PHP source code and step into the directory. If you haven’t deleted it from when you compiled PHP itself, it should be ready to go. Within the source code is a directory called ext, inside of which are separate directories for each of the extensions.

You are reading a book sample. This rest of this section is available in the full book.

Installing multiple versions of PHP

You are reading a book sample. This rest of this section is available in the full book.

PEAR and PECL

You are reading a book sample. This rest of this section is available in the full book.

Composer

You are reading a book sample. This rest of this section is available in the full book.

Symfony2 bundles

You are reading a book sample. This rest of this section is available in the full book.

Appendix B : File & data format libraries for PHP

This chapter lists extensions/libraries/bindings broken down by category that can be used to work with different file and data formats in PHP. Within each category you will find the name of the library, followed by a list of file types (specified by their common file extension for brevity) that it supports, and the main website for that library. Where there isn’t a common file extension (for example streaming data formats), the name of the format is used instead.

The list below is not exhaustive, and not all libraries allow you to both read and write the given formats, or support all features of the format. Check the relevant documentation for the library and try it out before selecting it for your project. If you haven’t found a library to help you with the file format you’re looking for, try using your favourite search engine! If you know of one or find one that’s not listed below, please e-mail me (author@phpbeyondtheweb.com).

You are reading a book sample. This rest of this section is available in the full book.

Office documents

You are reading a book sample. This rest of this section is available in the full book.

Compression, archiving & encryption

You are reading a book sample. This rest of this section is available in the full book.

Graphics

You are reading a book sample. This rest of this section is available in the full book.

Audio

You are reading a book sample. This rest of this section is available in the full book.

Multimedia & video

You are reading a book sample. This rest of this section is available in the full book.

Programming, technical and data interchange

You are reading a book sample. This rest of this section is available in the full book.

Misc

You are reading a book sample. This rest of this section is available in the full book.

Appendix C : Sources of help

Even with excellent books like this on on the market, you will sometimes need a little additional help when you come across a tricky problem with PHP. Below are some potential sources of help.

You are reading a book sample. This rest of this section is available in the full book.

The PHP manual

You are reading a book sample. This rest of this section is available in the full book.

Official mailing lists

You are reading a book sample. This rest of this section is available in the full book.

Stack Overflow

You are reading a book sample. This rest of this section is available in the full book.

Other books

You are reading a book sample. This rest of this section is available in the full book.

Newsgroups

You are reading a book sample. This rest of this section is available in the full book.

PHP Subredit

You are reading a book sample. This rest of this section is available in the full book.

PHP news sites

You are reading a book sample. This rest of this section is available in the full book.

Appendix D : Interesting libraries, tools, articles and projects

Throughout this book you will have seen links to numerous libraries and tools, related to the topic in hand. This Appendix lists many more PHP related projects which didn’t fit neatly into other sections of the book but are nevertheless interesting. Many show off the potential of PHP beyond just websites, and others provide libraries for tasks that are often performed with desktop, server or other longer-running software. Some are just cool! If you know of any that are relevant here, let me know and they can be included in future updates of the book.

You are reading a book sample. This rest of this section is available in the full book.

Alternative programming styles

You are reading a book sample. This rest of this section is available in the full book.

Machine learning, artificial intelligence and data analysis

You are reading a book sample. This rest of this section is available in the full book.

Databases

You are reading a book sample. This rest of this section is available in the full book.

Natural language

You are reading a book sample. This rest of this section is available in the full book.

Graphics and imaging

You are reading a book sample. This rest of this section is available in the full book.

Unicode

You are reading a book sample. This rest of this section is available in the full book.

Audio

You are reading a book sample. This rest of this section is available in the full book.

Event driven PHP

You are reading a book sample. This rest of this section is available in the full book.

PHP internals

You are reading a book sample. This rest of this section is available in the full book.

Website/service APIs

You are reading a book sample. This rest of this section is available in the full book.

You are reading a book sample. This rest of this section is available in the full book.

Javascript

You are reading a book sample. This rest of this section is available in the full book.

Servers

You are reading a book sample. This rest of this section is available in the full book.

Programming

You are reading a book sample. This rest of this section is available in the full book.

Financial

You are reading a book sample. This rest of this section is available in the full book.

Appendix E : Integrated Development Environments for PHP

The following is a list of IDE’s (Integrated Development Environments) aimed at the PHP market. The purpose is to help you explore the options available, none are specifically recommended for, or dedicated to, CLI/non-web programming. The list is broken into two parts, first the open source options and then the commerical offerings.

If you know of one that’s not listed below, please let me know (author@phpbeyondtheweb.com).

You are reading a book sample. This rest of this section is available in the full book.

Opensource

You are reading a book sample. This rest of this section is available in the full book.

Commercial

You are reading a book sample. This rest of this section is available in the full book.

Appendix F : Changelog

This book is an evolving project. This section will detail the changes made each time the book is updated, so if you’ve already read through a previous version of the book, you can see at a glance what has changed and what new information has been included.

Current Version : 1.0

Changes in Version 1.0

  • This is the first release, so everything is new!

PHP Beyond the web

Shell scripts, desktop software, system daemons and more ... without learning a new language

by Rob Aley

Small?1389009702

Minimum price:  $19.99
Suggested price:  $29.99