Posts Tagged: Zend Framework

Nov 08

Zend_Search_Lucene: Not enterprise-ready

Zend Framework has been attracting more and more attention from the PHP community lately, and while it lacks certain things (like code generation) that other frameworks (like Rails) have implemented to great effect, Zend Framework 2.0 is slowly taking shape and it looks like it will be the framework of choice for startups and enterprises alike. (Yes, it will even have code generation.)

But despite having several “enterprise-ready” components, I’ve found that one in particular is not: Zend_Search_Lucene, Zend Framework’s native PHP implementation of Apache Lucene, written in Java.

Don’t get me wrong; Zend_Search_Lucene is great for a small site or blog. However, from extensive personal experience, it is not appropriate for a site with a medium or large index. I think this should be noted upfront in the documentation.

Against my better judgment, the company I work for migrated our previous search solution to Zend_Search_Lucene. On pretty heavy-duty hardware, indexing a million documents took several hours, and searches were relatively slow. The indexing process consumed vast amounts of memory, and the indexes frequently became corrupted (using 1.5.2). A single wild card search literally brought the web server to its knees, so we disabled that feature. Memory usage was very high for searches, and as a result requests per second necessarily declined heavily as we had to reduce the number of Apache child processes.

We have since moved to Solr (a Lucene-based Java search server) and the difference is dramatic. Indexing now takes around 10 minutes and searches are lightning fast. What a difference a language makes.

Like this post? You might also like Coalmine, my centralized error tracking service for your apps. Coalmine captures errors and all kinds of helpful debugging information, notifies you, and makes it all searchable. Check it out!

Sep 08


On and off for the last two or three months, Jurriën Stutterheim and I wrote Zend_Paginator, the pagination component for Zend Framework. Yesterday it was officially released as part of Zend Framework 1.6.

This was our first contribution to the framework, and was very much a collaborative working relationship. I first created a proposal in about half an hour on Christmas Eve 2007 for how I thought a pagination component should work after being dissatisfied with existing solutions. It contained some good ideas about flexibility, but on its own it was half an idea, at best. So I promptly forgot about it.

Then Jurriën came along and created a proposal based on his Zym_Paginate component, which was part of Zym Framework, itself an extension of Zend Framework.

At the urging of Zend, we merged our proposals. We refined our ideas, incorporating aspects from each of our proposals in addition to brand new thoughts. Then I wrote a new component from scratch based in part on Jurriën’s existing work. He wrote the excellent unit tests (with 100% code coverage, not always easy to achieve!), and I wrote the DocBook documentation. I also can’t overstate the positive impact the community had, particularly from Bryce Lohr who prompted us to decouple and reorganize major portions of the component.

Anyway, I’m very happy with how it turned out. There are still a few issues, most of them minor—and because of the component’s flexible design, there are almost always workarounds. They should be fixed in time for 1.6.1.

Here’s some praise for the component:

Upcoming Zend_Paginator in 1.6 is absolutely brilliant. Wasted time be gone.
Joakim Nygård

The new Zend Framework 1.6 release candidate includes a Zend_Paginator class, which is an excellent thing to have around because I know I’ve re-invented that wheel on every site I’ve developed. My only criticism of the new Zend_Paginator is it offers a daunting amount of possibilities.
Chris Beer

This really is a beautifully crafted component, very elegant, very classy.
—David Mintz, via e-mail

I’ve seen dozens of different paginators in the last couple of years, and to be honest, this one is by far the best. You got everything right! I can’t wait for this component to hit the incubator.

It’s great, well done guys.
Federico Cargnelutti

Like this post? You might also like Coalmine, my centralized error tracking service for your apps. Coalmine captures errors and all kinds of helpful debugging information, notifies you, and makes it all searchable. Check it out!

Jun 08

The future of PHP

Jurriën Stutterheim posted an interesting article on the future of PHP recently (“PHP… what to say?”). In it he argues that PHP shouldn’t try to maintain complete backwards compatibility in the next release, saying,

“PHP 6 will make or break PHP as a language. For PHP to make it, it needs a clear vision of what it wants to be. Trying to maintain compatibility with ancient and mostly poorly written scripts can’t be part of this vision.”

The language can’t advance, he says, if it’s chained to outdated yet popular open-source projects.

But here’s the problem. If you’re intent on being the lingua franca of the Web, backwards compatibility is critical. It’s part of that compatibility-stability-security triumvirate that users love.

You see this conflict elsewhere in the tech industry. Microsoft’s greatest strength and biggest weakness is backwards compatibility. Users love it because decade-old programs still run in Windows. If you’re, say, a medical transcription company that uses an application written in the late ’90s, your company is going to stay with Windows indefinitely, because no external factor forces you to update that application. And Microsoft isn’t going to do anything to jeopardize that.

This is the same wall that PHP is headed toward. I view it as a wall. Others might not; it’s certainly not a bad place to be at the moment. On the plus side, it means PHP developers have more opportunities afforded to them because of the popularity of the language. The downside is that they can look left and right and see niche languages innovating in compelling ways. The more “cosmopolitan” PHP programmers inevitably develop language envy and move on, but these are exactly the kind of developers PHP should fight to retain.

It doesn’t have to be this way. PHP could flourish as a smaller language. This is the tack that Apple has taken, and it seems to work well for the company and its users. Apple is one of the most innovative companies in the entire tech industry owing in no small part to this strategy.

For my part, I’d like to see first-class functions and closures included in the language. An object-oriented API ($file->read(1024)) alongside the procedural functions (fread($resource, 1024)) would also be welcome.

But none of that will happen, because PHP is a language in decline. Not a decline in usage—it will only continue to expand its reach—but in the addition of innovative features from other languages. There will be no need to evolve; most of the agitators for change will have moved on.

There is one hope for PHP, however: Zend Framework. I think if it can gain a foothold among developers, some of these trends can be reversed. Hopefully (for PHP) everyone hasn’t moved on to Ruby by then.

Like this post? You might also like Coalmine, my centralized error tracking service for your apps. Coalmine captures errors and all kinds of helpful debugging information, notifies you, and makes it all searchable. Check it out!

Dec 06

Does Ajax have a place in the application framework?

“Where’s Ajax?” It’s one of the questions that keeps popping up in reference to Zend Framework. I suppose it’s human nature: most people just don’t get as excited about the fundamentals of the project as I do—the MVC implementation, the Lucene-derived search engine [update: not so much], the caching component. They want flashy stuff. They want Ajax. And they don’t want to work too hard for it.

In the last 10 months, application frameworks have scrambled to be Web 2.0-compatible by creating their own Ajax components. In doing so, they have failed to consider the issue of whether they should, and instead concerned themselves primarily with whether they could. Not surprisingly, this has led to questionable implementations.

To determine whether or not a change should be made in my own applications, I apply a rubric. First, is the change essential? If so, the choice is clear; do it. If not, I continue to ask questions. In which ways would the change be beneficial? How would it be harmful? And so on.

Naturally, this same set of questions is applicable to the topic at hand, so let’s examine each in turn.

First, is it essential? Application frameworks are designed for the server-side. What interaction does Ajax have with the server, besides requesting a given URL via GET, POST, or some other method, and interpreting the result? None; this process is the same as a standard Web request. Ajax, therefore, does nothing new or spectacular from the point of view of the server, so it’s not fulfilling a fundamental need.

Is it beneficial? Most Ajax components wrap JavaScript libraries like or Dojo, even to the point of using identical method names. For those with a basic understanding of JavaScript, how much time is saved here over using those libraries directly, really? For those without that understanding, is it wise to incorporate technology into your application without knowing how it works, at least broadly? Better to spend a day learning JavaScript and the principles behind XMLHttpRequest and then choose a JavaScript library that best matches your needs and programming style. Why write more lines in Java with Wicket when you could just write JavaScript directly?

As far as I can tell, most of these components exist simply so that developers can stay cozy and warm within the cocoon of their favorite language, without having to venture out into another one. This is a particularly common attitude among Java developers, as Google Web Toolkit illustrates. Although it has plenty of stated benefits (some of them of questionable validity), let’s be frank—the real reason to use it is to never have to write a line of JavaScript. Is JavaScript so bad that it’s better to have to compile it from some other language, especially when excellent debuggers like Firebug exist?

Is it harmful? One of the most basic tenets of modern web development is the separation of presentation and logic, but Ajax components often ignore this concept. Although some implementations involve the creation of entire forms from within the controller, let’s assume a more intelligent design, as a view helper.

Even with a view helper implementation, you’re suddenly presented with a problem: unless the code is prepared in some way prior to run-time (similar to a “compile” step), you’re now most likely mixing JavaScript event handling with your HTML. Some people don’t even realize that this is a problem. However, just as server-side logic should be separated from the view (as with a multitier architecture) and style should be separated from content (as with CSS), behavior has no place within the presentation. That is to say, JavaScript belongs in a separate, included file. It’s called “unobtrusive JavaScript“.

As typically implemented, Ajax components also have the practical effect of preventing those without JavaScript from using the application at all. That includes visitors using screen reading software, most visitors using some sort of mobile browser, and many corporate users straining under IT paranoia. Altogether, these can account for up to 5% of all visitors to a public website. If frameworks are generating inaccessible code, then it is just as much their fault as the fault of the developer using the tool.


It seems clear to me that Ajax components add no real value if the code is not prepared prior to run-time, and little value if it is. Instead of creating Ajax components, framework developers should let JavaScript library developers handle Ajax simplification and focus on other matters. JavaScript is good and it has its place, but not within the application framework.

Like this post? You might also like Coalmine, my centralized error tracking service for your apps. Coalmine captures errors and all kinds of helpful debugging information, notifies you, and makes it all searchable. Check it out!

Nov 06

Zend/PHP Conference wrap-up

It was a fun week. Of course, the big news on Tuesday was Zend’s new partnership with Microsoft, which promises to make PHP perform better with Windows in general, and IIS and Windows Server 2003 in particular. Zend’s own branch of PHP, Zend Core, will include even more improvements. Both Zend and Microsoft were obviously excited by the announcement, and I’ve got to admit that I was pretty impressed by the ease of configuring PHP in IIS 7. They showed some in-house benchmarks, but I’ll be interested to see some independent tests conducted when the final product ships (sometime next year?).

At some point during the same keynote, MySQL AB CEO Mårten Mickos jumped on stage for about seven seconds just to brag about what we already know—that MySQL is the most-used database with PHP. I wish he had spent a few more seconds, then, and explained why they clearly don’t bother to test their installations with PHP, since the default installation of MySQL prevents PHP from compiling if it also has OpenSSL (a somewhat common package).

In any event, the hot topics for this year were scalability and security, and there were lots of sessions on both. Eli White, the senior developer over at Digg, gave a great talk on scaling techniques (OpenOffice Impress format). George Schlossnagle, lead developer of APC and, frankly, entirely too many other PECL packages, followed up with his own excellent talk on scalability (PDF), which was naturally a bit more focused on caching.

Jaisen Mathai of FotoFlix briefly covered JSON and PHP, although (as with all of the sessions) the time constraints really limited how much he was able to talk about. After the session I mentioned Zend_Json to him—although any JSON library will do for encoding and decoding objects, I like the things attached to that one in particular, like Zend_Json_Server. In one of my projects I decided to expose entire PHP objects as JavaScript, modify them on the client using some voodoo, then return back the entire object. That’s the kind of thing that’s really useful when you’re dealing with complex data and needing a simple UI that doesn’t require lots of page refreshing, and although his talk was a good introduction to JSON/PHP interaction in general it didn’t cover interesting ways that JSON can be used outside of retrieving exposed, external information or your standard Ajax stuff. And to be honest, I felt like the examples in his presentation would have worked better as REST services. Oh well, you can only do so much with 45 minutes, and he was a pretty good presenter notwithstanding.

Unit testing also got a deserving nod from Sebastian Bergmann, creator of PHPUnit. His talk hammered home the test-first methodology and also revealed some new functionality in the upcoming PHPUnit 3, including being able to automatically run Selenium tests. That’s hot.

I should also mention that Chris Anderson, author of The Long Tail, gave an excellent keynote on Wednesday. It wasn’t particularly tailored to apply to PHP, though, as one questioner pointed out. Still, I’m just as (if not more) interested in business strategy as I am in software development, so I loved it. Plus, we all got free hardcover copies of his new book, so that was nice (I picked up two—my boss wanted one). Maybe it’s just because he’s been touting the concept for awhile, but I was impressed at how on the ball he was with the Q&A session at the end.

Andrei Zmievski of Yahoo!, though, won the prize for having far and away the most interesting session of the week—”Unicoding with PHP” (PDF). He walked through in great detail the challenges of creating a Unicode-aware PHP 6 that (almost) transparently handles the hellish details of the standard. I’m guessing the Japanese user-base is drooling in anticipation of being able to (among other things) use hiragana, katakana, and kanji to represent variables instead of the Latin alphabet. Andrei promised a PHP 6 Unicode pre-release to get the functionality out there and tested, so keep an eye out for that. I’ll link to it when it’s available. Update: Here it is.

Last but not least, the Zend Framework get-together was one of the highlights of the week. Meeting Gavin, Darby, Bill, Andi, and all the rest of the Zend crew (along with contributors and users like Richard and Keith) in person was great, and I got a t-shirt. BONUS.

Anyway, that’s enough for this post, and I haven’t even covered half of it. Really, if you haven’t been before and you’re interested in PHP, you really should make some time to go next year. Plus, you know… t-shirts!

Like this post? You might also like Coalmine, my centralized error tracking service for your apps. Coalmine captures errors and all kinds of helpful debugging information, notifies you, and makes it all searchable. Check it out!