Tim's Weblog
Tim Strehle’s links and thoughts on Web apps, software development and Digital Asset Management, since 2002.
2016-02-24

A dropdown for large lists in a Symfony 3 form with choice_loader and Select2

I’m currently learning the PHP framework Symfony 3. The Symfony documentation is excellent for a quick start, but the first serious HTML form I wanted to build touched some ill-documented parts, made me file a bogus bug report (sorry guys) and took me about 20 hours to implement. So here’s how I did it. I hope that helps if you want to do something similar.

The problem was that I needed a dropdown that scales well to huge numbers of options – i.e. the dropdown shouldn’t require listing all available options (there might be thousands), but let the user search instead. And let her enter new values as well.

The UI side of the problem is already solved by the highly recommended Select2, “the jQuery replacement for select boxes”. My finished example form looks like this, it just lets you pick and add tags and dumps all form data when the form is submitted:

To build this, first install Select2 by adding these lines to your Symfony app’s composer.json file under the "require" key (and running php composer.phar update):

        "components/jquery": "~2.1",
        "select2/select2": "~4.0",

To make jQuery and Select2 available below the DocumentRoot, add symlinks (mind you, I’m a Symfony newbie and there’s probably a better way to do this):

$ cd web/
$ ln -s ../vendor/components/jquery
$ ln -s ../vendor/select2/select2/dist select2

Now create a Twig template that initializes the Select2 element in “tags” mode and sets up the Ajax request:

{# app/Resources/views/default/new.html.twig #}

{% extends "base.html.twig" %}

{% block stylesheets %}

  {{ parent() }}

  <link href="{{ asset('select2/css/select2.min.css') }}" type="text/css" rel="stylesheet" media="screen" />    

{% endblock %}

{% block body %}

  <div class="container">

    {{ form_start(form) }}
    {{ form_widget(form) }}
    {{ form_end(form) }}

    {{ msg }}
  
  </div>

{% endblock %}

{% block javascripts %}

  {{ parent() }}
  
  <script src="{{ asset('jquery/jquery.min.js') }}"></script>
  <script src="{{ asset('select2/js/select2.min.js') }}"></script>
  
  <script type="text/javascript">
    
    $(function() 
    {
        // Called by Select2 to determine autocomplete request 
        // query string parameters
        
        var getAjaxListData = function(params)
        {
            return {
                q: params.term,
                p: params.page
            };
        };
        
        // Turn the plain SELECT input into a Select2 element
        
        $('#form_tags').select2(
        {
            ajax:
            {
                url: '{{ path('ajax_tag_list') }}',
                dataType: 'json',
                delay: 250,
                data: getAjaxListData,
                cache: true,
                tags: true
            }
        });
    });
    
  </script>
  
{% endblock %}

The form data class that’ll hold the submitted form data is pretty short, it just defines a property for each form field:

<?php
// src/AppBundle/Entity/Task.php

namespace AppBundle\Entity;

class Task
{
    public $title = '';
    public $tags = [ ];
}

Nothing special so far. But then it took me a while to figure out how to use the ChoiceType field’s choice_loader option (which is mostly undocumented except for a “only needed in advanced cases” warning) to load just the first n (say, 10 or 50) options from a potentially very large list in the database. And how to make it properly validate and re-display options which are not in the initially loaded list subset. It seems that this requires a Form Event handler that catches selected options on form submit and feeds them into the ChoiceLoaderInterface implementation before it generates its list.

That’s why the Symfony controller passes the form builder instance to the choice loader:

<?php
// src/AppBundle/Controller/DefaultController.php

namespace AppBundle\Controller;

use AppBundle\Entity\Task;
use AppBundle\ChoiceLoader;
use Sensio\Bundle\FrameworkExtraBundle\Configuration\Route;
use Symfony\Bundle\FrameworkBundle\Controller\Controller;
use Symfony\Component\HttpFoundation\Request;
use Symfony\Component\HttpFoundation\JsonResponse;
use Symfony\Component\Form\Extension\Core\Type\SubmitType;
use Symfony\Component\Form\Extension\Core\Type\ChoiceType;
use Symfony\Component\Form\Extension\Core\Type\TextType;

class DefaultController extends Controller
{
    /**
     * Show the form
     *
     * @Route("/", name="homepage")
     */
    public function newAction(Request $request)
    {
        // Create the form
        
        $task = new Task();
        $formbuilder = $this->createFormBuilder($task);

        $form = $formbuilder
            ->add
            (
                'title', 
                TextType::class, 
                [ 
                    'required' => true,
                    'attr' => [ 'style' => 'width: 400px;' ]
                ]
            )
            ->add
            (
                'tags', 
                ChoiceType::class, 
                [ 
                    'multiple' => true, 
                    'required' => true,
                    'attr' => [ 'style' => 'width: 400px;' ],
                    'choice_loader' => new ChoiceLoader($formbuilder)
                ]
            )
            ->add
            (
                'save', 
                SubmitType::class, 
                [ 
                    'label' => 'Submit'
                ]
            )
            ->getForm();

        $form->handleRequest($request);

        // Dump submitted form data

        $msg = 'not submitted';

        if ($form->isSubmitted() && $form->isValid()) 
        {
            $msg = print_r($task, true);
        }

        return $this->render('default/new.html.twig', 
        [
            'form' => $form->createView(),
            'msg' => $msg
        ]);
    }

    /**
     * Ajax request from Select2 autocomplete
     *
     * @Route("/ajax_tag_list", name="ajax_tag_list")
     */
    public function getTagListAction(Request $request)
    {
        // The search term from the autocomplete input
        $query = $request->query->get('q', '');
        
        $choice_loader = new ChoiceLoader(false);
        $choice_list = $choice_loader->getChoicesList($query);
        $list_values = [ ];
        
        // Dummy entry for the search term, allows creating new values

        if (strlen($query) > 0)
        {
            $list_values[ ] =
            [
                'id' => 'create:' . $query,
                'text' => $query . ' *'
            ];
        }
        
        // Add regular entries
        
        foreach ($choice_list as $label => $id)
        {
            $list_values[ ] =
            [   
                'id' => $id,
                'text' => $label
            ];
        }
                
        $response = new JsonResponse();
        $response->setData([ 'results' => $list_values ]);
        
        return $response;
    }
}

And finally, the ChoiceLoader magic that does autocomplete, validation, and creates new choices. The example form holds the list of available choices in the session data, you’ll want to use some database instead:

<?php
// src/AppBundle/ChoiceLoader.php

namespace AppBundle;

use Symfony\Component\Form\ChoiceList\ArrayChoiceList;
use Symfony\Component\Form\ChoiceList\Loader\ChoiceLoaderInterface;
use Symfony\Component\Form\FormBuilderInterface;
use Symfony\Component\Form\FormEvent;
use Symfony\Component\Form\FormEvents;

class ChoiceLoader implements ChoiceLoaderInterface
{
    // Currently selected choices
    protected $selected = [ ];
    
    /**
     * Constructor
     */
    public function __construct($builder)
    {
        if (is_object($builder) && ($builder instanceof FormBuilderInterface))
        {
            // Let the form builder notify us about initial/submitted choices

            $builder->addEventListener
            (
                FormEvents::POST_SET_DATA, 
                [ $this, 'onFormPostSetData' ]
            );

            $builder->addEventListener
            (
                FormEvents::POST_SUBMIT, 
                [ $this, 'onFormPostSetData' ]
            );
        }
    }
    
    /**
     * Form submit event callback
     * Here we get notified about the submitted choices.
     * Remember them so we can add them in loadChoiceList().
     */
    public function onFormPostSetData(FormEvent $event)
    {
        $this->selected = [ ];
        
        $formdata = $event->getData();
        
        if (! is_object($formdata))
        {
            return;
        }
        
        $this->selected = $formdata->tags;
    }

    /**
     * Choices to be displayed in the SELECT element.
     * It's okay to not return all available choices, but the
     * selected/submitted choices (model values) must be
     * included.
     * Required by ChoiceLoaderInterface.
     */
    public function loadChoiceList($value = null)
    {
        // Get first n choices
        
        $choices = $this->getChoicesList(false);

        // Check which choices are missing

        $missing_choices = array_flip($this->selected);

        foreach ($choices as $label => $id)
        {
            if (isset($missing_choices[ $id ]))
            {
                unset($missing_choices[ $id ]);
            }
        }
        
        // Now add selected choices if they're missing

        foreach (array_keys($missing_choices) as $id)
        {
            $label = $this->getChoiceLabel($id);
            
            if (strlen($label) === 0)
            {
                continue;
            }

            $choices[ $label ] = $id;
        }
        
        return new ArrayChoiceList($choices);
    }

    /**
     * Validate submitted choices, and turn them from strings
     * (HTML option values) into other datatypes if needed
     * (not needed here since our choices are strings).
     * We're also using this place for creating new choices
     * from new values typed into the autocomplete field.
     * Required by ChoiceLoaderInterface.
     */
    public function loadChoicesForValues(array $values, $value = null)
    {
        $result = [ ];
        
        foreach ($values as $id)
        {
            if (substr($id, 0, 7) === 'create:')
            {
                $label = substr($id, 7);
                $result[ ] = $this->createChoice($label);
            }
            elseif ($this->choiceExists($id))
            {
                $result[ ] = $id;
            }
        }

        return $result;
    }

    /**
     * Turn choices from other datatypes into strings (HTML option
     * values) if needed - we can simply return the choices as 
     * they're strings already.
     * Required by ChoiceLoaderInterface.
     */
    public function loadValuesForChoices(array $choices, $value = null)
    {
        $result = [ ];
        
        foreach ($choices as $id)
        {
            if ($this->choiceExists($id))
            {
                $result[ ] = $id;
            }
        }

        return $result;
    }
    
    /**
     * Get first n choices
     */
    public function getChoicesList($filter)
    {
        // Init our dummy list - not needed if you're
        // working with a proper database
        $this->initChoices();
        ksort($_SESSION[ 'tag_choices' ]);
        
        // Get choices list from the session; you'll use
        // something like SQL here instead
        
        $result = [ ];
        $cnt = 0;
        $limit = 10;
        $filter = mb_strtolower($filter);
        $filter_len = mb_strlen($filter);
        
        foreach ($_SESSION[ 'tag_choices' ] as $label => $id)
        {
            if ($filter_len > 0)
            {
                if (mb_substr(mb_strtolower($label), 0, $filter_len) !== $filter)
                {
                    continue;
                }
            }
            
            $result[ $label ] = $id;
            
            if (++$cnt >= $limit)
            {
                break;
            }
        }
        
        return $result;
    }
    
    /**
     * Validate whether a choice exists
     */
    protected function choiceExists($id)
    {
        // Init our dummy list - not needed if you're
        // working with a proper database
        $this->initChoices();

        // Check whether this choice exists in the session; 
        // you'll use something like SQL here instead

        $label = array_search($id, $_SESSION[ 'tag_choices' ], true);

        return 
        (
            $label === false 
            ? false 
            : true
        );
    }

    /**
     * Get choice label
     */
    protected function getChoiceLabel($id)
    {
        // Init our dummy list - not needed if you're
        // working with a proper database
        $this->initChoices();

        // Get choice from the session; 
        // you'll use something like SQL here instead

        $label = array_search($id, $_SESSION[ 'tag_choices' ], true);

        return 
        (
            $label === false 
            ? false 
            : $label
        );
    }

    /**
     * Create a new choice
     */
    protected function createChoice($label)
    {
        // Init our dummy list - not needed if you're
        // working with a proper database
        $this->initChoices();

        // Add choice to the session; 
        // you'll use something like SQL here instead
        
        $id = sprintf
        (
            'choice%s (%s)', 
            count($_SESSION[ 'tag_choices' ]), 
            $label
        );
        
        $_SESSION[ 'tag_choices' ][ $label ] = $id;
        
        return $id;
    }
    
    /**
     * Initialize a list of dummy choices
     */
    protected function initChoices()
    {
        if (isset($_SESSION[ 'tag_choices' ]))
        {
            return;
        }

        $_SESSION[ 'tag_choices' ] = [ ];
        
        for ($code = 65; $code <= 90; $code++)
        {
            $id = chr($code);
            $label = $id . ' tag';
            
            $_SESSION[ 'tag_choices' ][ $label ] = $id;
        }
    }
}

If you have suggestions for improving this code, please let me know!

Update (2016-04-04): I added a listener for FormEvents::POST_SET_DATA so that the tag values are properly populated not only on form submit, but also when initially displaying the form with existing values.

Update (2016-07-12): Seems Symfony 3.2 (due later this year) could make this easier with the new CallbackChoiceLoader class, see New in Symfony 3.2: Lazy loading of form choices.