One Year Later: Reflections on Developing with Extbase and Fluid

A Year LaterReflections on Developing with ExtBase and Fluid

Date: June 10, 2011

Author: Zach Davis, Cast Iron Coding, Inc.

Zach Davis - June 10, 2011 - San Francisco TYPO3 ConferenceSlide #1

Friday, June 10, 2011


•Located in Portland, Oregon

•Founded in 2004 with two employees; now at seven employees (six

developers) plus local sub-contractors

•95% of all sites we build are built with TYPO3

•While we are a full-service agency, CIC’s core competency is

development; we have developed hundreds of TYPO3 extensions for

our clients.

•Our clients are drawn from a variety of industries but we have some

specialization in non-profit and higher education sites

About Cast Iron Coding and how we came to TYPO3

Background

Background

About CIC, about this talk

The goal of today’s talk is to share some of our experiences with ExtBase/Fluid and to discuss specific uses of ExtBase/Fluid that have been helpful on recent projects.



•Low quality extension—hard to configure, lack of Typoscript abstraction, presentation layer hard-coded into code—

extensions in the extension library meant that for most projects we ended up developing custom extensions for clients.

•Consequently, the cost associated with pretty standard features was often too high for clients, especially for smaller clients

on tight budgets

•This made it increasingly more difficult for us to compete with Wordpress , which has increasingly come to be seen as a

viable CMS, on smallish (under 15k) projects.

•Instead of hoping for better extensions in the TER, we chose to devote time to finding a way to a) reduce the time required to

develop extensions and b) increase the amount of code that was reusable between projects (or make sure we were writing

code that was focused on the domain)

In 2008, we needed a way to reduce development time and costs to remain competitive with other open source systems such as Wordpress (for smaller projects) and Drupal or Joomla (for larger projects)

There were a number of challenges that we experienced working with TYPO3 in 2008 and 2009:

CICMVC: Our first attempt at MVC in TYPO3

CICMVC

Our first attempt at MVC in TYPO3



In response to these challenges, CIC developed CICMVC to replace piBase.

CICMVC is a light-weight, high-performance MVC layer that sits on top of piBase. Some parts of it worked really well for us:


CICMVC


•Few constraints, which isn’t always a bad thing! Because we can ensure quality control in-house, we didn’t need to include

the sorts of constraints that are present in FLUID and ExtBase.

•Therefore, CICMVC uses PHP views—not so popular these days, perhaps, but not much more syntactically difficult than

FLUID, and quite a bit faster.

•Core classes of CICMVC include: controller, model, helper, view, peer, query, form, requests, request filters, exceptions, email

and partials.

•Built-in models and peers for common objects (Digital asset and digital asset peer, user models, countries and states, etc).

•Separate pi classes (pi1, pi2, etc) and page types are used for different output modes (HTML, JSON, XML, etc).




CICMVC


•Database accessed by way of “peer” classes which relied on query objects. In most cases, records could be received easily

without writing much SQL . The query object used arrays to store the query components. In more complicated cases, the

query object and the peer class offer easy, direct access to the underlying TYPO3 DB class. We _never_ ran into performance

problems in relation to database access.

•Basic security accomplished with basic request filters. Variables coming from GET or POST need to be defined as strings,

integers, etc, and are validated as such before being passed to controllers. For most extensions, this was sufficient.

•CICMVC was not an academic exercise. It was a real-world attempt to dramatically reduce development time in order to

increase company revenue while also reducing client cost

•More lightweight than ExtBase, feels significantly faster and more responsive.

•The year after we started using CICMVC widely across all projects, we saw revenue increase by about 75% while the

company stayed relatively small. Without wanting to mistake correlation for causation, our strong sense is that CICMVC had a

huge effect on CIC’s bottom line.

Some parts of it worked really well for us:




CICMVC


•Lack of an ORM meant that persisting objects from the TYPO3 front-end was never as easy as it should have been. Instead of

persisting objects automatically, we built a form class that made error handling, validation, etc easier. (Creating and persisting

objects from a front-end form was the developer’s responsibility—not the framework’s—which made sense because there

often is not a simple 1-1 relationship between form and model.)

•Because it was a custom solution without a community behind it, our clients had to place a great deal of trust in their long-

term relationship with CIC in order to use CICMVC as the foundation for a large project.

•CICMVC was not an academic exercise. If developers weren’t careful, business logic could wind up in a view or presentation

logic could wind up in a model. Lack of constraints required CIC to place a lot of trust in its developers, even with internal QA

processes.

•Limited feature set and lack of configurability

•Lack of modularity: view helpers were stored in one large “helper” class rather than as individual classes

•Magic getters and setters on models were both a blessing and a curse

Other parts weren’t so great...


Zach Davis - June 10, 2011 - San Francisco TYPO3 Conference

CICMVC

A few examples

Slide #7

Some CICMVC ExamplesEmory Center for the Study of Law and Religion


http://cslr.law.emory.edu/people/person/name/broyde/




CICMVC

A few examples

Slide #8

Some CICMVC ExamplesCarnegie Corporation of New York


http://carnegie.org/programs/international-peace-and-security/

http://carnegie.org/programs/international-peace-and-security/


CICMVC

A few examples

Slide #9

Some CICMVC ExamplesCarnegie Corporation of New York

Models and

Peers

Modules

PublicationViews



We make the leap

Goodbye CICMVC, hello ExtBase!

In mid-2010, we drop CICMVC and switch to ExtBase

Oracle Storage Backend Fluid Page Templates Backend Reporting Tools

Custom ExtBase Content Elements

Tides: Project Directory and SFDC

Tides: Impact Stories





We make the leap

•We wanted clients to have a solutions that other TYPO3 developers could support. It’s already a challenge to sell US

customers on TYPO3 with only a handful of vendors here; further lock-in due to a custom extension framework makes that

pitch even more difficult.

•We wanted to give back to the TYPO3 project, which we didn’t feel was possible with CICMVC. Originally, before ExtBase,

we thought we might be able to share CICMVC eventually. However, once ExtBase took off, it was clear that another MVC

solution wouldn’t be helpful.

•We bought into the thinking behind Domain Driven Design. Staying current in terms of best practices and writing code that

can be changed without extensive refactoring keeps us competitive. DDD pushes us to think more conceptually, which

makes us better developers (and, IMO, better thinkers!)

We had a number of reasons for making the leap





We make the leap

•Performance, performance, performance! Would our clients perceive ExtBase extensions as slow and under performing?

•We were concerned about decreased productivity. The kickstarter wasn’t quite there yet, which meant having to do more

work to setup tables. Writing getters and setters on models is a pain. Fluid syntax took some getting used to and writing view

helpers can be time consuming.

•We were concerned that ExtBase wasn’t stable enough, even in late 2010, and that APIs would change after we’d done a

project.

•We worried that the persistence layer was too restrictive, and that some of the approaches expected by DDD were not well

suited for real-world applications.

•We suspected that ExtBase would not play nice with other key TYPO3 features, especially localization and Workspaces.

And some concerns...



Our first attempt: ExtBase content elements

ExtBase Content Elements

Hard to do with piBase, easy with ExtBase

We thought we’d take something difficult—creating editor-friendly, custom content elements—and see if we could make it easier by using ExtBase



Sidebar CalloutContent Element





Custom content elements defined in ext_tables.phpext_tables.php:

ext_localconf.php:





Define actions in ContentController class





Setup a Fluid view for the content element





Our first attempt: ExtBase content elementsUsing the same approach, we created a handful of similar elements.





•Editors LOVE these content elements. Without much work on our part, clients get a library of custom content elements that

force them to be consistent without being overly constrained.

•After you create the first one, additional custom content elements are very cost effective. We include a placeholder

extension called “t3site” in our starter package that contains a controller for custom content elements and a sample custom

content element. The ext_tables config can be easily reproduced and new content elements can be created that use the

same controller.

•Someday, we should create a model for tt_content records that could be shared across these different content element

types.

•We save time (thereby making our work more profitable) because we don’t have to set up classes in the RTE that allow

editors to create similar element with <P> tags and whatnot, and because we don’t have to map and maintain a bunch of

FCEs.

•Editors get a better user interface because instead of flexforms, they get standard TYPO3 tt_content records with the

conventional fields that they’re used to.

Our first attempt: ExtBase content elementsWe learned a few things from these custom content elements





Our second attempt: a standard extBase search/listplug-inStructurally similar to the Carnegie publications plug-in, Tides project directory and impact stories seemed like a good candidate for moving away from CICMVC

•Built around quite a few simple models: Constituents, GeographicScopes, Issues, Programs, Projects, Strategies, Services,

and Stories

•Data for project directory is stored in Salesforce, pulled into TYPO3 every couple hours via Salesforce’s SOAP web service

• Story views include issue list, list of stories, search results (for AJAX requests), single story, and story teaser box

•Project directory views include list of projects, lat/long coordinates (for Google Maps request, output as JSON), search

results (for AJAX requests), and single project.

ExtBase Front-end Plugins

A typical ExtBase use case



Tides Project Directory




http://www.tides.org/community/project-directory/

http://www.tides.org/community/project-directory/


Tides Issues Stories




http://www.tides.org/impact/stories/

http://www.tides.org/impact/stories/


Our second attempt: a standard extBase search/listplug-in

One of ExtBase’s most important (and overlooked) capabilities is that it can unify back-end and front-end functionality. There’s some hackery involved, but using the same repositories in both sides of TYPO3 as well as ExtBase’s persistence layer saves considerable time.




https://github.com/castiron/tides.org/blob/master/Source/typo3conf/ext/tidesprojdir/Classes/Task/ProjectImportLogic.php




Our second attempt: a standard extBase search/listplug-inNonetheless, ExtBase’s persistence layer wasn’t sufficient for handling the complex queries (not all that complex, perhaps?) generated by the search interface.




https://github.com/castiron/tides.org/blob/master/Source/typo3conf/ext/tidesprojdir/Classes/Domain/Repository/ProjectRepository.php





Oracle Storage Backend for ExtBase

It worked!

Our third attempt: Oracle ExtBase Storage Backend Client needed a searchable white paper/video library. We couldn’t, in good conscience, create piBase extensions for this, and felt that we had to use ExtBase. One requirement was that TYPO3 be running on an Oracle DB


http://www.pyramis.com/us/market-insights.html




•Out of the box, DBAL pretty much ignores ExtBase because ExtBase pretty much ignores the TYPO3 DB class. Instead of using

methods like exec_SELECTquery, the ExtBase persistence layer uses passes SQL directly to the sql_query method on TYPO3’s

DB class. DBAL, as I understand it, doesn’t really parse these queries.

•So, instead of trying to use these DB methods, we followed ExtBase’s example and made a OracleDBBackend Class that

generates queries that conform to Oracle’s query syntax.

•Because the objects don’t change on the front-end, we didn’t implement addRow, updateRow or other save-related

methods. Those are on our roadmap

•Setting a different storage backend for Extbase in ext_localconf isn’t hard (and while it works, this probably shouldn’t be

considered a public API):

Our approach:

Our third attempt: Oracle ExtBase Persistence Layer


It worked!



•Avoiding DBAL as much as possible is definitely the way to go. Instead of making DBAL smarter about parsing ExtBase

queries, it’s easier to just make ExtBase generate syntactically correct Oracle queries.

•We ran into more problems with TYPO3/Oracle/DBAL in general than we did with ExtBase specifically, once we

implemented our storage backend.

•Performance, while not great, is tolerable. Because of the keyword search, the output is essentially uncached

•Work involved with this was really not all that difficult. We probably spent about 2 developer days on this.

•Not ExtBase related, per se, but worth mentioning: if TYPO3 is to truly succeed as an “enterprise” CMS, it needs better

Oracle support. Developers who have worked on DBAL have done a great job, but there’s more to do. Would be nice if

companies using TYPO3 and Oracle could pool resources.

What we learned:

Our third attempt: Oracle ExtBase Persistence Layer


It worked!



•Uses FLUID for page templates

•Does not require mapping mechanisms like TV, automaketemplate, parts and subparts, and other approaches

•Instead, Typoscript Objects are made available to the Fluid page template, just as objects are made available to fluid

templates in ExtBase

•Editors use backend layouts for the editing interface, which is more consistent with TYPO3’s overall back-end UI than TV

•FCEs are no longer needed because we have custom content elements, which are just as easy to setup, easier to use, and

easier to track in Git

•We’re looking for a column solution (like Bernhard’s nested content elements extension or Snowflake’s columns extension in

the TER).

•Relationship between tt_content and pages is obscured in XML, which makes some tasks difficult

We’ve never been entirely satisfied with TemplaVoila. FCEs mean flexforms, and flexforms provide an inconsistent UI for editors. TV stores configuration (mapping) in the DB, so we can’t easily track it in Git. The classic page module and it’s limitations was a non-starter.

We needed a new approach, so we started building the fluid pages extension.

Our fourth attempt: Fluid Page Templates

Fluid Page Templates

Lots of possibilities for continued development



Our fourth attempt: Fluid Page Templates

Fluid Page Templates

Lots of possibilities for continued development


https://github.com/castiron/t3seed/blob/master/Source/typo3conf/ext/t3site/Sites/Main/Resources/Private/Layouts/main.html

https://github.com/castiron/t3seed/blob/master/Source/typo3conf/ext/t3site/Sites/Main/Resources/Private/Layouts/main.html


Our fifth attempt: Reports

Reports

A little ExtBase, a lot of ExtJs and ExtDirect



Conclusions


•On low to medium traffic sites, with relatively flat data, ExtBase is clearly a better choice than piBase

•Even with more complex data or with larger sets of data, there’s a good chance that you can circumvent ExtBase’s storage

backend in your repositories by writing your own queries, especially if the application is read-only on the front-end.

•We get greater code reusability in ExtBase as we add models, services, and view helpers to our own ExtBase helper

extension.

•We think that Fluid, in particular, should be more widely used throughout TYPO3. It’s silly (and super-confusing for

newcomers) that we have so many templating methods in TYPO3 (TemplaVoila, parts/subparts/markers, and FLUID). TYPO3

should try to standardize on FLUID and continue to develop the FLUID TEMPLATE cObj so that it can be more useful as a tool

for page templates.

Overall, results are positive. We’ll continue using ExtBase!

Conclusions!



Conclusions


•Sharing models and repositories between the front-end and the back-end can be a huge time saver, especially when you

have to import data into TYPO3 from an external system for display on the front-end

•As recent discussions have shown, the persistence layer is a real pain point for ExtBase developers. Given TYPO3’s

idiosyncrasies, I have my doubts about the possibilities offered by Doctrine. Instead, I’d like it to be EASIER for developers to

get low-level database access and to create objects out of query results in the repositories (thereby moving away from the

one-to-one model / DB table relationship).

Overall, results are positive. We’ll continue using ExtBase!

Conclusions!


Technology

One Year Later: Reflections on Developing with Extbase and Fluid