October 2016

Saturday, October 22, 2016

Using Action-Domain-Responder on the command line

Yesterday I was working on a project in Radar and needed to create a command line tool for it. #

In the past, I’ve always used Symfony Console which I like. Since my application was already built using Radar and adhering to Action-Domain-Responder and Clean Architecture, I wanted a solution that was more consistent with the rest of the codebase. #

The selling point to Clean Architecture is that your domain logic is separate from the delivery mechanism. So if Radar was my delivery mechanism, I should be able to have a different command line delivery mechanism and my domain shouldn’t know or care. #

I took a look at Aura.Cli which seems nice. However, it’s very much a command line library. Out of the box, it’s not a framework for building command line tools adhering to ADR. #

So I took a stab at writing one (that uses Aura.Cli), it’s called Cadre.CliAdr. #

At this point, it’s more of a proof of concept rather than a production solution. It’s heavily based on Radar and uses the same patterns. #

Here is an example of how you’d use it. It will seem very familiar if you’ve used Radar. #

use Aura\Di\ContainerBuilder;
use Aura\Cli\CliFactory;
require __DIR__ . '/../vendor/autoload.php';
$di = (new ContainerBuilder())
$adr = $di->get('cadre:cliadr/adr');
$factory = new CliFactory();
$context = $factory->newContext($GLOBALS);
$stdio = $factory->newStdio();
$adr->route('test', function ($params) {
	return 'This is a test.';
exit($adr->run($context, $stdio));

Expect revisions and documentation to come. In the meantime, please post any comments here or as an issue on GitHub. #

Thursday, October 20, 2016

Introducing Cadre.Module

Today I published a new component Cadre.Module. #

This component was born out of my side project that’s using Radar. #

Stock Radar #

Radar is built around Aura.Di which is a very nice dependency injection container. If you’re interested in learning more about Radar check out Radar Under the Hood. #

One thing that’s great about Aura.Di is the concept of ContainerConfig objects. #

Here is an example: #

use Aura\Di\ContainerBuilder;
$container_builder = new ContainerBuilder();
// use the builder to create and configure a container
// using an array of ContainerConfig classes
$di = $container_builder->newConfiguredInstance([

This is nice because I can package my DI configuration into smaller classes that configure a single thing. An example of this could be one ContainerConfig that configures Twig and another that configured Atlas.Orm. #

The problem I ran into was where to put things if they are related to different areas. Also, there was an issue about how to make things work between dev and production if they needed a different configuration. #

The specific use case that prompted the development of this library was configuring PHP Debug Bar which I’ve been working with a lot lately. #

PHP Debug Bar can collect data from many different sources. I’m using it currently with Twig and Atlas.Orm. When I push my project to production, I will not want to be loading PHP Debug Bar, but it’s very useful during development. #

It’s also possible that I’ll want to reuse this code in the future on projects that may not use Twig and/or Atlas.Orm. #

Cadre.Module #

In Cadre.Module I introduce the Module and ModuleLoader classes. Both are useable as ContainerConfig objects. Modules define four additional methods that are inspired by Composer. #

Each module can define what other modules they require, require in dev, conflict with and replace. #

I started out also implementing provide and suggest, but decided that they had limited to no application in this use case. #

While the new modules are just ContainerConfigs with extra metadata, the module loader does the heavy lifting. #

While still behaving as a ContainerConfig, a module loader starts out with a list of modules in the same way composer.json defines your starting packages. #

When it’s used, it goes through and loads each of the modules and loads their required modules. It also checks for conflicts and replacements along the way. #

The define and modify methods just proxy through to all of the loaded modules. #

The other neat feature of Cadre.Module is that modules can query the module loader to see if another module is loaded. #

This is especially useful for optional modules or development modules that may not always be present. #

Conclusion #

I’m interested in what you think of Cadre.Module. Check out the documentation and ask questions either here in the comments or as an issue on GitHub. #

Tuesday, October 18, 2016

Tracking Multiple Connections with PHP Debug Bar and Atlas ORM

On Friday I talked about a new library I created that helps integrate Atlas.ORM with PHP Debug Bar. #

Hari K T replied asking about using multiple connections. For instance, if you use Atlas with a default master database for writes and multiple read-only slave databases. My library AtlasOrm.DebugBar.Bridge does not support this is any way out of the box. #

I spent the weekend thinking about how to solve this. #

First, I asked “How would I do this right now with what’s available?” and the answer was: #

$atlasContainer = new Cadre\AtlasOrmDebugBarBridge\AtlasContainer(
$debugbar = new DebugBar\StandardDebugBar();
$atlasCollector = new Cadre\AtlasOrmDebugBarBridge\AtlasOrmCollector($atlasContainer);
$atlasContainer->setReadConnection('readonly', function () use ($atlasCollector) {
	$readOnly = new Cadre\AtlasOrmDebugBarBridge\ExtendedPdo(
	$atlasCollector->addConnection('readonly', $readOnly->getPdo());
	return $readOnly;

I wasn’t happy with this solution. Too complicated and too easy to mess up. #

It was difficult, because of Atlas.ORM inheriting the Aura.SQL requirement that adding read and write connections require a callable that returns the connection. This is done because if we want to register multiple possible connections, we don’t want to connect to all of them right away. We only want to connect when we’re using the connection. #

Finally, yesterday morning I came up with a possible solution. #

If I created a new class ConnectionFactory that could be configured identically as a connection and be passed into AtlasContainer as a callable (I implement the __invoke method). It could then also be aware of the AtlasOrmCollector which would have a new method addConnectionFactory that registered itself with the ConnectionFactory object. #

This way, I could easily configure new connections (via ConnectionFactory), add them to the AtlasContainer and AtlasOrmCollector and they would wire themselves up. The TraceablePDO connection would only get added to the collector when it was invoked which prevented me from invoking all of the connections up front whether they were used or not. #

The new method of adding multiple connections is then: #

$atlasContainer = Cadre\AtlasOrmDebugBarBridge\AtlasContainer(
$factory = new Cadre\AtlasOrmDebugBarBridge\ConnectionFactory(
$atlasContainer->setReadConnection('readonly', $factory);
$collector = new Cadre\AtlasOrmDebugBarBridge\AtlasOrmCollector($container);
$collector->addConnectionFactory($factory, 'readonly');
$debugbar = new DebugBar\StandardDebugBar();

I think it looks a lot cleaner and less prone to errors. #

This code is currently sitting as a PR to allow any interested parties to comment before it gets merged. Introducing ConnectionFactory. Feel free to review and comment. #

Friday, October 14, 2016

Collecting Data from Atlas ORM with PHP Debug Bar

In my last article, I talked about how I found an N+1 bug in Atlas ORM. #

I had mentioned how it took a little work to get PHP Debug Bar configured with Atlas but didn’t really explain why it was difficult, or how I got them working together. #

At first, it seemed like it would be easy. Debug Bar comes with a PDOCollector and Atlas is based on PDO. In Atlas\Orm\AtlasContainer it creates a new Aura\Sql\ConnectionLocator then creates and adds an Aura\Sql\ExtendedPdo connection to it. #

PHP Debug Bar requires you wrap your PDO connection in DebugBar\DataCollector\PDO\TraceablePDO which extends PDO. #

Aura\Sql\ExtendedPdo wraps a PDO connection and extends PDO however DebugBar\DataCollector\PDO\TraceablePDO is not a Aura\Sql\ExtendedPdo object. #

That means that I need to wrap PDO in DebugBar\DataCollector\PDO\TraceablePDO which is then wrapped in Aura\Sql\ExtendedPdo. #

There is no easy way of doing that without replacing some code. #

The first step is to create a new ExtendedPdo class that extends Aura\Sql\ExtendedPdo that overrides connect() which instantiates the PDO connection. We want to wrap the newly created connection in DebugBar\DataCollector\PDO\TraceablePDO. #

Next, we create a new AtlasContainer class that extends Atlas\Orm\AtlasContainer that overrides setConnectionLocator to instantiates my new ExtendedPdo instead of Aura\Sql\ExtendedPdo. #

Finally, we can use the DebugBar\DataCollector\PDO\PDOCollector which requires a DebugBar\DataCollector\PDO\TraceablePDO connection. We can get that out of our AtlasContainer like this: #

$pdo = $atlasContainer->getConnectionLocator()->getDefault()->getPdo();

To make this step easier I created AtlasOrmCollector which extends DebugBar\DataCollector\PDO\PDOCollector but expects my AtlasContainer instead of a PDO connection in the constructor. This way I can pull out the DebugBar\DataCollector\PDO\TraceablePDO connection and pass it to the parent constructor. #

I’ve packaged all of this up into the library AtlasOrm.DebugBar.Bridge which is also available on packagist as cadre/atlasorm-debugbar-bridge. #

You can now integrate PHP Debug Bar into your applications that use Atlas ORM as simply as this: #

$atlasContainer = new Cadre\AtlasOrmDebugBarBridge\AtlasContainer(
$debugbar = new DebugBar\StandardDebugBar();
	new Cadre\AtlasOrmDebugBarBridge\AtlasOrmCollector($atlasContainer)

Monday, October 10, 2016

Complex Database Relationships with AtlasORM

In my side project I’m working with a very well normalized database with many relationships. #

One of the most complex pages is a Creator where it shows info about the creator and all of the works that they have contributed to. #


This is the database structure for this page. On the left, we have a Creator which has a one-to-many relationship to seven different join tables. Each of these tables represents this creators credit on that particular type of work. So if the creator worked on an RPG book there would be a RpgBookCreator that shows what credit that creator had in a particular RPG book. A credit could be something like Author or Illustrator. #

When I first started working on this project I was doing plain SQL with PDO. Quickly I realized that for all the pages I was building I’d be writing a whole lot of queries and that didn’t sound particularly fun. #

Then I tried porting part of this page over to Doctrine 2 ORM and that was a huge mess. All of the mapping files and entity objects was taking a long time to get right. Then, once I had it configured for Creator, RpgBookCreator, RpgBook, Credit, GameLine and Publisher (which is only 1/7 of the structure) I ran the query and the page just sat there spinning. I was not familiar enough with Doctrine to make it perform the way I wanted it to. #

It was suffering from the “N+1 Problem”: #

This problem occurs when the code needs to load the children of a parent-child relationship (the “many” in the “one-to-many”). Most ORMs have lazy-loading enabled by default, so queries are issued for the parent record, and then one query for EACH child record. As you can expect, doing N+1 queries instead of a single query will flood your database with queries, which is something we can and should avoid. #

Well, it just so happens Paul M. Jones the creator of Radar wrote the book “Solving The N+1 Problem in PHP”. Not only that, he is the creator of AtlasORM described as: #

Atlas is a data mapper implementation for your persistence model (not your domain model). #

If anyone would have an ORM that could handle my data structure, it’s Paul. #

It didn’t take very long to get my site working with Atlas. There is a very useful CLI that will analyze your database and generate the scaffolding needed for Atlas to work. #

The two types of classes required for atlas are Tables and Mappers. The Table classes (ex: CreatorTable) described things like the column names, the primary key, and the default values. #

The Mapper describes the relationships between tables. By default, the method setRelated is empty because the CLI doesn’t create them. However, it was very easy for me to write a script that did automatically create these relationships because I was using a consistent naming scheme. #

Here is the final CreatorMapper which describes all of its one-to-many relationships. #

Finally, in my CreatorsAtlasRepository class I query the database, specifying all of the relationships I’d like to include. #

It was pretty fast. I picked a creator with many credits across different types of work (Monte Cook) and it loaded in about 1.8 seconds. However, I wanted to make sure Atlas was querying efficiently. #

In an ideal situation, it would query each table once or join some tables and have less than 20 queries. I wasn’t expecting this. #

In a more expected situation, it would query the tables close to the root once and some of the edge tables (publisher, game_line) might get queries a few times for different batches of data. Probably around 30-40 queries depending on the creator. #

I found a project called PHP Debug Bar that could hook into different systems (including PDO) and show stats for each page. #

It took a little work to get that configured with Atlas, but I got it working and went to see how many queries were made to render the Monte Cook page. #

I was very surprised to see a total of 484 queries. That was an order of magnitude larger than I was expecting. Clearly, there was still an N+1 problem with Atlas. I dug through the query log and determined that the issue was with the edge relationships. There were many many queries for single publishers and game lines. #

I submitted a PR explaining my issue. Today Paul commented that he had a patch in a development branch and wanted me to test it. #

I’m happy to report that the same Monte Cook page now loads with only 32 queries which is right where I was expecting it to be. The page is a little faster at 1.5 seconds which is good enough for development. Before the site goes live I’ll implement a caching layer. #

This patch should get merged into the main branch soon. This is really a fabulous library that I’m planning on using in all of my new projects. #

Tuesday, October 4, 2016

LazyArray Officially Part of Aura.Di

In my prior posts about integrating Symfony Forms with Radar, I created a helper class called LazyArray. #

It was designed so I could pass an array of lazily instantiated objects into a setter like Twigs setExtensions method. #

Paul liked my implementation and asked me to submit a PR. #

Today it was merged and included in Aura.Di 3.2.0 which makes me very excited. #

Published by Andrew Shell on and last updated .