The evolution of a module

Software modules evolve with user demand and more opportunities to connect with other modules. The evolution is good, including the occasional rewrite to create a better structure for future evolution.

Competing against the evolutionary development is the demand to leap major barriers including the need to comply with major changes to the environment. The conversion from Drupal 6 to Drupal 7 is one barrier facing many module developers. They need to leap the barrier fast because most new Web site development uses Drupal 7.

Drupal 7 offers the opportunity to convert from conventional database usage into the Drupal entity system, something that might bring better database compatibility and faster connection to other software modules. The conversion from regular flexible data to Drupal entities is another barrier to evolution because of the huge volume of work required and the instant lack of compatibility with Drupal 6. You can no longer maintain similar code between the Drupal 7 version of your module and the Drupal 6 version.

The IP Login module is an example of a typical Drupal 6 module evolving while leaping into Drupal 7. Some of the design decisions make the evolution easier, including not using all the new features of Drupal 7.

The module has a significant usage in Drupal 6 and only a small uptake on Drupal 7. The developers need to maintain compatibility to avoid the overhead of separate development. Avoiding the glossy new features of Drupal 7 limits the code changes and make it easier to maintain the two versions in parallel.

There are some serious design changes proposed for flexibility, performance, and future compatibility. I contributed an idea or two plus some code examples. There are at least two issues open to track the changes. The end result will be a giant leap forward.

All the changes are compatible across Drupal 6 and 7. The only requirement for easy code maintenance is to stay away from the Drupal 7 entity swamp.

The module has a lookup table that needs to be fast. The data in the table is entered by the user and designed for flexibility. The critical lookup field can be made faster by devolving the data into raw numbers. You store the user data in one table then generate the lookup table from the user data, converting the user data into a form optimised for lookup.

Take an example. The critical lookup is an IP address match. The lookup might search for 127.54.38.1. The user input covering the address might be 127.54.30-35.*. You have to make several searches to find the user input covering the address.

A solution is to store the user input in one table then generate an address range in a lookup table. The lookup table would have a from address and a to address. The range, for our example 127.54.30-35.*, would be from 127.54.30.0 to 127.54.35.255. The generation of ranges from input would be rare. The lookup of ranges frequent. The result would be a big improvement for the most frequently used part of the code.

Some of the input items will generate multiple lookup ranges in the lookup table. You need a ling back from the lookup table items to the data generating the lookup entries. The easiest link is the user id for the input. When you edit the IP addresses for a user, you delete all input and lookups for the user then insert new lists.

You could use Drupal 7 entities to store the user data and the lookup table. The entity system makes sense for the user data because the data is connected to the user. The entity system is too slow for the lookup table and the lookup table does not have to load user data the way an entity connection would load user data. The lookup only needs the id of a user then Drupal can take over the loading of the user data when needed by other modules.

Within Drupal 7 entities are several opportunities that are also traps when misused. Using the IP Login example, the IP range entries could be in a multiple entry field instead of a single string. The multiple entry field looks nice and gives you some advantages but every entry is an extra overhead for every use of that user and slows down overall user processing.

Conclusion

The module IP lookup can improve speed by evolving the lookup process and avoid major slowdown by avoiding the Drupal 7 entity processing. Evolution is good. Drupal 7 is both an opportunity to evolve and a barrier to evolution. Minimise the conversion work by avoiding new Drupal 7 features until the conversion is finished then selectively choose only those new Drupal 7 features which help the evolution.