Query builder
Table of Contents
The query builder provides a set of methods to create queries programmatically.
This chapter provides examples of the most common queries.
Warning
From a security point of view, the documentation of ->createNamedParameter() and ->quoteIdentifier() are an absolute must read and follow section. Make very sure you understand this and use it for each and every query to prevent SQL injections!
The query builder comes with a happy little list of small methods:
- Set type of query:
->select,() ->count,() ->update,() ->insertand() ->delete() - Prepare
WHEREconditions - Manipulate default
WHERErestrictions added by TYPO3 for->select() - Add
LIMIT,GROUP BYand other SQL functions executeexecutes aQuery () SELECTquery and returns a result, a\Doctrine\objectDBAL\ Result executeexecutes anStatement () INSERT,UPDATEorDELETEstatement and returns the number of affected rows.
Most of the query builder methods provide a fluent interface, return an instance of the current query builder itself, and can be chained:
$queryBuilder
->select('uid')
->from('pages');
Instantiation
To create an instance of the query builder, call
Connection and pass the table as an
argument. The ConnectionPool object can be
injected via dependency injection.
<?php
declare(strict_types=1);
namespace MyVendor\MyExtension\Domain\Repository;
use TYPO3\CMS\Core\Database\ConnectionPool;
final class MyRepository
{
public function __construct(
private readonly ConnectionPool $connectionPool,
) {}
public function findSomething()
{
$queryBuilder = $this->connectionPool
->getQueryBuilderForTable('aTable');
}
}
Attention
Never instantiate and initialize the query builder manually using
dependency injection or
General, otherwise you
will miss essential dependencies and runtime setup.
Warning
The QueryBuilder holds internal state and must not be reused for
different queries. In addition, a reuse comes with a
significant performance penalty and memory consumption.
Use one query builder per query. Get a fresh one by calling
$connection->create if the same table is
involved, or use
$connection for a
query to a different table. Do not worry, creating those object instances
is quite fast.
select() and addSelect()
Create a
SELECT query.
Select all fields:
// SELECT *
$queryBuilder->select('*')
->select and a number of other methods of the query builder
are variadic
and can handle any number of arguments. In
->select each argument
is interpreted as a single field name to be selected:
// SELECT `uid`, `pid`, `aField`
$queryBuilder->select('uid', 'pid', 'aField');
Argument unpacking can be used if the list of fields already is available as array:
// SELECT `uid`, `pid`, `aField`, `anotherField`
$fields = ['uid', 'pid', 'aField', 'anotherField'];
$queryBuilder->select(...$fields);
->select automatically supports
AS and quotes identifiers. This
can be especially useful for
join operations:
// SELECT `tt_content`.`bodytext` AS `t1`.`text`
$queryBuilder->select('tt_content.bodytext AS t1.text')
With
->select the list of fields to be selected is specified, and with
->add further elements can be added to an existing list.
Mind that
->select replaces any formerly registered list instead of
appending it. Thus, it is not very useful to call
select twice in a
code flow or after an
->add. The methods
->where and
->and share the same behavior:
->where replaces all
formerly registered constraints,
->and appends additional
constraints.
A useful combination of
->select and
->add can be:
$queryBuilder->select(...$defaultList);
if ($needAdditionalFields) {
$queryBuilder->addSelect(...$additionalFields);
}
Calling the
execute function on a
->select query returns
a result object of type
\Doctrine\. To receive single rows, a
->fetch loop is used on that object, or
->fetch to return a single array with all rows. A typical
code flow of a
SELECT query looks like this:
// use TYPO3\CMS\Core\Database\Connection;
$queryBuilder = $this->connectionPool->getQueryBuilderForTable('tt_content');
$result = $queryBuilder
->select('uid', 'header', 'bodytext')
->from('tt_content')
->where(
$queryBuilder->expr()->eq('bodytext', $queryBuilder->createNamedParameter('lorem', Connection::PARAM_STR))
)
->executeQuery();
while ($row = $result->fetchAssociative()) {
// Do something with that single row
debug($row);
}
Read how to correctly instantiate a query builder with the connection pool. See available parameter types.
Default Restrictions
Note
->select and
->count queries trigger TYPO3 magic that adds
further default where clauses if the queried table is also registered via
$GLOBALS. See the RestrictionBuilder section for details on that topic.
count()
Create a
COUNT query, a typical usage:
// use TYPO3\CMS\Core\Database\Connection;
// SELECT COUNT(`uid`) FROM `tt_content` WHERE (`bodytext` = 'lorem')
// AND ((`tt_content`.`deleted` = 0) AND (`tt_content`.`hidden` = 0)
// AND (`tt_content`.`starttime` <= 1669885410)
// AND ((`tt_content`.`endtime` = 0) OR (`tt_content`.`endtime` > 1669885410)))
$queryBuilder = $this->connectionPool->getQueryBuilderForTable('tt_content');
$count = $queryBuilder
->count('uid')
->from('tt_content')
->where(
$queryBuilder->expr()->eq('bodytext', $queryBuilder->createNamedParameter('lorem', Connection::PARAM_STR))
)
->executeQuery()
->fetchOne();
Read how to correctly instantiate a query builder with the connection pool. See available parameter types.
Remarks:
- Similar to the
->selectquery type,() ->countautomatically triggers the magic of the RestrictionBuilder that adds default restrictions such as() deleted,hidden,starttimeandendtimewhen defined in TCA. - Similar to
->selectquery types,() ->executewithQuery () ->countreturns a result object of type() \Doctrine\. To fetch the number of rows directly, useDBAL\ Result ->fetch.One () - The first argument to
->countis required, typically() ->countor(*) ->countis used, the field name is automatically quoted.('uid') -
There is no support for
DISTINCT, instead a->grouphas to be used, for example:By () // Equivalent to: // SELECT DISTINCT some_field, another_field FROM my_table $queryBuilder ->select('some_field', 'another_field') ->from('my_table') ->groupBy('some_field') ->addGroupBy('another_field');Copied! - If
->countis combined with() ->group, the result may return multiple rows. The order of those rows depends on the used DBMS. Therefore, to ensure the same order of result rows on multiple different databases, aBy () ->groupshould always be combined with anBy () ->order.By ()
delete()
Create a
DELETE FROM query. The method requires the table name from which
data is to be deleted. Classic usage:
// use TYPO3\CMS\Core\Database\Connection;
// DELETE FROM `tt_content` WHERE `bodytext` = 'lorem'
$queryBuilder = $this->connectionPool->getQueryBuilderForTable('tt_content');
$affectedRows = $queryBuilder
->delete('tt_content')
->where(
$queryBuilder->expr()->eq('bodytext', $queryBuilder->createNamedParameter('lorem', Connection::PARAM_STR))
)
->executeStatement();
Read how to correctly instantiate a query builder with the connection pool. See available parameter types.
Remarks:
- For simple cases it is often easier to write and read using the
->deletemethod of the Connection object.() - In contrast to
->select,() ->deletedoes not automatically add() WHERErestrictions likeAND `deleted` = 0. ->deletedoes not magically transform a() DELETE FROM `tt_content` WHERE `uid` = 4711into something likeUPDATE `tt_content` SET `deleted` = 1 WHERE `uid` = 4711internally. A soft-delete must be handled at application level with a dedicated lookup in$GLOBALSto check if a specific table can handle the soft-delete, together with an['TCA'] ['the Table'] ['ctrl'] ['delete'] ->updateinstead.() - Deleting from multiple tables at once is not supported:
DELETE FROM `table1`, `table2`can not be created. ->deleteignores() ->join() ->deleteignores() set:Max Results () DELETEwithLIMITdoes not work.
update() and set()
Create an
UPDATE query. Typical usage:
// use TYPO3\CMS\Core\Database\Connection;
// UPDATE `tt_content` SET `bodytext` = 'dolor' WHERE `bodytext` = 'lorem'
$queryBuilder = $this->connectionPool->getQueryBuilderForTable('tt_content');
$queryBuilder
->update('tt_content')
->where(
$queryBuilder->expr()->eq('bodytext', $queryBuilder->createNamedParameter('lorem', Connection::PARAM_STR))
)
->set('bodytext', 'dolor')
->executeStatement();
Read how to correctly instantiate a query builder with the connection pool. See available parameter types.
->update requires the table to update as the first argument and a table
alias (for example,
t) as optional second argument. The table alias can
then be used in
->set and
->where expressions:
// use TYPO3\CMS\Core\Database\Connection;
// UPDATE `tt_content` `t` SET `t`.`bodytext` = 'dolor' WHERE `t`.`bodytext` = 'lorem'
$queryBuilder = $this->connectionPool->getQueryBuilderForTable('tt_content');
$queryBuilder
->update('tt_content', 't')
->where(
$queryBuilder->expr()->eq('t.bodytext', $queryBuilder->createNamedParameter('lorem', Connection::PARAM_STR))
)
->set('t.bodytext', 'dolor')
->executeStatement();
Read how to correctly instantiate a query builder with the connection pool. See available parameter types.
->set requires a field name as the first argument and automatically
quotes it internally. The second mandatory argument is the value to set a field
to. The value is automatically transformed to a named parameter of a prepared
statement. This way,
->set key/value pairs are automatically SQL
protected from injection by default.
If a field should be set to the value of another field from the row, quoting
must be turned off and
->quote and
false have to
be used:
// use TYPO3\CMS\Core\Database\Connection;
// UPDATE `tt_content` SET `bodytext` = `header` WHERE `bodytext` = 'lorem'
$queryBuilder = $this->connectionPool->getQueryBuilderForTable('tt_content');
$queryBuilder
->update('tt_content')
->where(
$queryBuilder->expr()->eq('bodytext', $queryBuilder->createNamedParameter('lorem', Connection::PARAM_STR))
)
->set('bodytext', $queryBuilder->quoteIdentifier('header'), false)
->executeStatement();
Read how to correctly instantiate a query builder with the connection pool. See available parameter types.
Remarks:
- For simple cases it is often easier to use the
->updatemethod of the Connection object.() ->setcan be called multiple times if multiple fields should be updated.() ->setrequires a field name as the first argument and automatically quotes it internally.() ->setrequires the value to which a field is to be set as the second parameter.() ->updateignores() ->joinand() ->set.Max Results () - The API does not magically add
deleted = 0or other restrictions, as is currently the case with select, for example. (See also RestrictionBuilder).
insert() and values()
Create an
INSERT query. Typical usage:
// INSERT INTO `tt_content` (`bodytext`, `header`) VALUES ('lorem', 'dolor')
$queryBuilder = $this->connectionPool->getQueryBuilderForTable('tt_content');
$affectedRows = $queryBuilder
->insert('tt_content')
->values([
'bodytext' => 'lorem',
'header' => 'dolor',
])
->executeStatement();
Read how to correctly instantiate a query builder with the connection pool.
Remarks:
- The
uidof the created database row can be fetched from the connection by using $queryBuilder->getConnection()->lastInsertId(). ->valuesexpects an array of key/value pairs. Both keys (field names / identifiers) and values are automatically quoted. In rare cases, quoting of values can be turned off by setting the second argument to() false. Then quoting must be done manually, typically by using->createon the values.Named Parameter () ->executeafterStatement () ->insertreturns the number of inserted rows, which is typically() 1.- An alternative to using the query builder for inserting data is using the
Connection object with its
->insertmethod.() - The query builder does not provide a method for inserting multiple rows
at once, use
->bulkof the Connection object instead to achieve that.Insert ()
from()
->from is essential for
->select and
->count query
types.
->from requires a table name and an optional alias name. The
method is usually called once per query creation and the table name is usually
the same as the one passed to
->get. If the query
joins multiple tables, the argument should be the name of the first table within
the
->join chain:
// FROM `myTable`
$queryBuilder->from('myTable');
// FROM `myTable` AS `anAlias`
$queryBuilder->from('myTable', 'anAlias');
->from can be called multiple times and will create the Cartesian
product of tables if not constrained by a respective
->where or
->and expression. In general, it is a good idea to use
->from only once per query and instead model the selection of multiple
tables with an explicit
->join.
where(), andWhere() and orWhere()
The three methods are used to create
WHERE restrictions for
SELECT,
COUNT,
UPDATE and
DELETE query types. Each argument is
usually an ExpressionBuilder object that
is converted to a string on
->execute or
->execute:
// use TYPO3\CMS\Core\Database\Connection;
// SELECT `uid`, `header`, `bodytext`
// FROM `tt_content`
// WHERE
// (
// ((`bodytext` = 'lorem') AND (`header` = 'a name'))
// OR (`bodytext` = 'dolor') OR (`bodytext` = 'hans')
// )
// AND (`pid` = 42)
// AND ... RestrictionBuilder TCA restrictions ...
$queryBuilder = $this->connectionPool->getQueryBuilderForTable('tt_content');
$result = $queryBuilder
->select('uid', 'header', 'bodytext')
->from('tt_content')
->where(
$queryBuilder->expr()->eq('bodytext', $queryBuilder->createNamedParameter('lorem', Connection::PARAM_STR)),
$queryBuilder->expr()->eq('header', $queryBuilder->createNamedParameter('a name', Connection::PARAM_STR))
)
->orWhere(
$queryBuilder->expr()->eq('bodytext', $queryBuilder->createNamedParameter('dolor')),
$queryBuilder->expr()->eq('bodytext', $queryBuilder->createNamedParameter('hans'))
)
->andWhere(
$queryBuilder->expr()->eq('pid', $queryBuilder->createNamedParameter(42, Connection::PARAM_INT))
)
->executeQuery();
Read how to correctly instantiate a query builder with the connection pool. See available parameter types.
Note the parenthesis of the above example:
->and encapsulates both
->where and
->or with an additional restriction.
Argument unpacking can become handy with these methods:
// use TYPO3\CMS\Core\Database\Connection;
$whereExpressions = [
$queryBuilder->expr()->eq('bodytext', $queryBuilder->createNamedParameter('lorem', Connection::PARAM_STR)),
$queryBuilder->expr()->eq('header', $queryBuilder->createNamedParameter('a name', Connection::PARAM_STR))
];
if ($needsAdditionalExpression) {
$whereExpressions[] = $someAdditionalExpression;
}
$queryBuilder->where(...$whereExpressions);
See available parameter types.
Remarks:
- The three methods are variadic.
They can handle any number of arguments. For instance, if
->wherereceives four arguments, they are handled as single expressions, all combined with() AND. - createNamedParameter is used to create a placeholder for a field value of a prepared statement. Always use this when dealing with user input in expressions to protect the statement from SQL injections.
->whereshould be called only once per query and resets all previously set() ->where,() ->andandWhere () ->orexpressions. AWhere () ->wherecall after a previous() ->where,() ->andorWhere () ->orusually indicates a bug or a rather weird code flow. Doing so is discouraged.Where () - When creating complex
WHERErestrictions,->getandSQL () ->getare helpful debugging tools to verify parenthesis and single query parts.Parameters () - If only
->eqexpressions are used, it is often easier to switch to the according method of the Connection object to simplify quoting and improve readability.() - It is possible to feed the methods directly with strings, but this is discouraged and usually used only in rare cases where expression strings are created in a different place that can not be easily resolved.
join(), innerJoin(), rightJoin() and leftJoin()
Joining multiple tables in a
->select or
->count query is done
with one of these methods. Multiple joins are supported by calling the methods
more than once. All methods require four arguments: The name of the table on the
left (or its alias), the name of the table on the right, an alias for the name
of the table on the right, and the join restriction as fourth argument:
// use TYPO3\CMS\Core\Database\Connection;
// SELECT `sys_language`.`uid`, `sys_language`.`title`
// FROM `sys_language`
// INNER JOIN `pages` `p`
// ON `p`.`sys_language_uid` = `sys_language`.`uid`
// WHERE
// (`p`.`uid` = 42)
// AND (
// (`p`.`deleted` = 0)
// AND (
// (`sys_language`.`hidden` = 0) AND (`overlay`.`hidden` = 0)
// )
// AND (`p`.`starttime` <= 1475591280)
// AND ((`p`.`endtime` = 0) OR (`overlay`.`endtime` > 1475591280))
// )
$queryBuilder = $this->connectionPool->getQueryBuilderForTable('sys_language')
$result = $queryBuilder
->select('sys_language.uid', 'sys_language.title')
->from('sys_language')
->join(
'sys_language',
'pages',
'p',
$queryBuilder->expr()->eq('p.sys_language_uid', $queryBuilder->quoteIdentifier('sys_language.uid'))
)
->where(
$queryBuilder->expr()->eq('p.uid', $queryBuilder->createNamedParameter(42, Connection::PARAM_INT))
)
->executeQuery();
Read how to correctly instantiate a query builder with the connection pool. See available parameter types.
Notes to the example above:
- The query operates with the
sys_table as the main table, this table name is given tolanguage get.Query Builder For Table () - The query joins the
pagestable asINNER JOINand gives it the aliasp. - The join condition is
`p`.`sys_language_uid` = `sys_language`.`uid`. It would have been identical to swap the expression arguments of the fourth->joinargument() ->eq.('sys_ language. uid', $query Builder->quote Identifier ('p. sys_ language_ uid')) - The second argument of the join expression instructs the
ExpressionBuilder to quote the value as
a field identifier (a field name, here a combination of table and field
name). Using createNamedParameter
would lead in quoting as value (
'instead of`in MySQL) and the query would fail. - The alias
p- the third argument of the->joincall - does not necessarily have to be set to a different name than the table name itself here. It is sufficient to use() pagesas third argument and not to specify any other name. Aliases are mostly useful when a join to the same table is needed:SELECT `something` FROM `tt_content` JOIN `tt_content` `content2` ON .... Aliases are also useful to increase the readability of->whereexpressions.() - The RestrictionBuilder has added
additional
WHEREconditions for both tables involved! Thesys_table obviously only specifies alanguage 'disabled' => 'hidden'asenablein its TCA ctrl section, while theColumns pagestable specifies the fieldsdeleted,hidden,starttimeandstoptime.
A more complex example with two joins. The first join points to the first table, again using an alias to resolve a language overlay scenario. The second join uses the alias of the first join target as left side:
// use TYPO3\CMS\Core\Database\Connection;
// SELECT `tt_content_orig`.`sys_language_uid`
// FROM `tt_content`
// INNER JOIN `tt_content` `tt_content_orig` ON `tt_content`.`t3_origuid` = `tt_content_orig`.`uid`
// INNER JOIN `sys_language` `sys_language` ON `tt_content_orig`.`sys_language_uid` = `sys_language`.`uid`
// WHERE
// (`tt_content`.`colPos` = 1)
// AND (`tt_content`.`pid` = 42)
// AND (`tt_content`.`sys_language_uid` = 2)
// AND ... RestrictionBuilder TCA restrictions for tables tt_content and sys_language ...
// GROUP BY `tt_content_orig`.`sys_language_uid`
$queryBuilder = $this->connectionPool->getQueryBuilderForTable('sys_language')
$constraints = [
$queryBuilder->expr()->eq('tt_content.colPos', $queryBuilder->createNamedParameter(1, Connection::PARAM_INT)),
$queryBuilder->expr()->eq('tt_content.pid', $queryBuilder->createNamedParameter(42, Connection::PARAM_INT)),
$queryBuilder->expr()->eq('tt_content.sys_language_uid', $queryBuilder->createNamedParameter(2, Connection::PARAM_INT)),
];
$queryBuilder
->select('tt_content_orig.sys_language_uid')
->from('tt_content')
->join(
'tt_content',
'tt_content',
'tt_content_orig',
$queryBuilder->expr()->eq(
'tt_content.t3_origuid',
$queryBuilder->quoteIdentifier('tt_content_orig.uid')
)
)
->join(
'tt_content_orig',
'sys_language',
'sys_language',
$queryBuilder->expr()->eq(
'tt_content_orig.sys_language_uid',
$queryBuilder->quoteIdentifier('sys_language.uid')
)
)
->where(...$constraints)
->groupBy('tt_content_orig.sys_language_uid')
->executeQuery();
Read how to correctly instantiate a query builder with the connection pool. See available parameter types.
Further remarks:
->joinand() innerare identical. They create anJoin INNER JOINquery, this is identical to aJOINquery.->leftcreates aJoin () LEFT JOINquery, this is identical to aLEFT OUTER JOINquery.->rightcreates aJoin () RIGHT JOINquery, this is identical to aRIGHT OUTER JOINquery.- Calls to
joinmethods are only considered for() ->selectand() ->counttype queries.() ->delete,() ->insertand() updatedo not support joins, these query parts are ignored and do not end up in the final statement.() - The argument of
->getshould be the leftmost main table.Query Builder For Table () - Joining two tables that are configured to different connections will throw an exception. This restricts the tables that can be configured for different database endpoints. It is possible to test the connection objects of the involved tables for equality and implement a fallback logic in PHP if they are different.
- Doctrine DBAL does not support the use of join methods in combination with
->update,() ->insertand() ->deletemethods, because such a statement is not cross-platform compatible.() -
Multiple join condition expressions can be resolved as strings like:
$joinConditionExpression = $queryBuilder->expr()->and( $queryBuilder->expr()->eq( 'tt_content_orig.sys_language_uid', $queryBuilder->quoteIdentifier('sys_language.uid') ), $queryBuilder->expr()->eq( 'tt_content_orig.sys_language_uid', $queryBuilder->quoteIdentifier('sys_language.uid') ), ); $queryBuilder->leftJoin( 'tt_content_orig', 'sys_language', 'sys_language', (string)$joinConditionExpression );Copied!
orderBy() and addOrderBy()
Add
ORDER BY to a
->select statement. Both
->order and
->add require a field name as first argument:
// SELECT * FROM `sys_language` ORDER BY `sorting` ASC
$queryBuilder = $this->connectionPool->getQueryBuilderForTable('sys_language');
$queryBuilder->getRestrictions()->removeAll();
$languageRecords = $queryBuilder
->select('*')
->from('sys_language')
->orderBy('sorting')
->executeQuery()
->fetchAllAssociative();
Read how to correctly instantiate a query builder with the connection pool.
Remarks:
->orderresets all previously specified orders. It makes no sense to call this function again after a previousBy () ->orderorBy () ->add.Order By () - Both methods need a field name or a
table.or afield Name tableas first argument. In the example above the call toAlias. field Name ->orderwould have been identical. All identifiers are quoted automatically.By ('sys_ language. sorting') - The second, optional argument of both methods specifies the sort order. The
two allowed values are
'ASC'and'DESC', where'ASC'is default and can be omitted. - To create a chain of orders, use
->orderand then multipleBy () ->addcalls. The call toOrder By () ->ordercreatesBy ('header')->add Order By ('bodytext')->add Order By ('uid', 'DESC') ORDER BY `header` ASC, `bodytext` ASC, `uid` DESC - To achieve more complex sortings, which can't be created with QueryBuilder,
you can fall back on the underlying raw Doctrine QueryBuilder. This is
accessible with
->get. It doesn't do any quoting, so you can do something likeConcrete Query Builder () $concrete. Make sure to quote properly as this is entirely your responsibility with the Doctrine QueryBuilder!Query Builder->order By ('FIELD (eventtype, 0, 4, 1, 2, 3)');
groupBy() and addGroupBy()
Add
GROUP BY to a
->select statement. Each argument of the
methods is a single identifier:
// GROUP BY `pages`.`sys_language_uid`, `sys_language`.`uid`
->groupBy('pages.sys_language_uid', 'sys_language.uid');
Remarks:
- Similar to
->selectand() ->where, both methods are variadic and take any number of arguments, argument unpacking is supported:() ->groupBy (...$my Group Array) - Each argument is either a direct field name
GROUP BY `bodytext`, atable.or afield Name tableand is properly quoted.Alias. field Name ->groupresets all previously defined group specification and should only be called once per statement.By () - For more complex statements you can use the raw Doctrine QueryBuilder. See remarks for orderBy()
union() and addUnion()
Method union provides a streamlined way to combine result sets from multiple
queries.
union(string |Query Builder $part) - Creates the initial
UNIONquery part by accepting either a raw SQL string or aQueryinstance. CallingBuilder unionresets all previous union definitions, it should therefore only be called once, using() addto add subsequent union parts.Union () addUnion (string |Query Builder $part, Union Type $type = Union Type:: DISTINCT) -
Adds additional
UNIONparts to the query. The$typeparameter accepts:UnionType:: DISTINCT - Combines results while eliminating duplicates.
UnionType:: ALL - Combines results and retains all duplicates. Not removing duplicates can be a performance improvement.
Note
While technically possible, it is not recommended to send direct SQL queries
as strings to the union and add methods. We recommend to use a
query builder.
If you decide to do so you must take care of quoting, escaping, and valid SQL Syntax for the database system in question. The Default Restrictions are not applied on that part.
Named placeholders, such as created by
Query
must be created on the outer most QueryBuilder See the example below.
See also
- W3School: SQL UNION Operator
- For technical details see the changelog entry Feature: #104631 - Add UNION Clause support to the QueryBuilder.
Database provider support of union() and addUnion()
Query can be used create
UNION clause queries not compatible with all database providers,
for example using
LIMIT/ in each part query or other stuff.
When building functional tests, run them on all database types that should be supported.
Example using union() on two QueryBuilders
<?php
namespace MyExtension\MyVendor\Service;
use Doctrine\DBAL\Query\UnionType;
use TYPO3\CMS\Core\Database\Connection;
use TYPO3\CMS\Core\Database\ConnectionPool;
use TYPO3\CMS\Core\Database\Query\QueryBuilder;
final readonly class MyService
{
public function __construct(
private ConnectionPool $connectionPool,
) {}
public function getTitlesOfSubpagesAndContent(
int $parentId,
): ?array {
$connection = $this->connectionPool->getConnectionForTable('pages');
$unionQueryBuilder = $connection->createQueryBuilder();
// Passing the outermost QueryBuilder to the subqueries
$firstPartQueryBuilder = $this->getUnionPart1QueryBuilder($connection, $unionQueryBuilder, $parentId);
$secondPartQueryBuilder = $this->getUnionPart2QueryBuilder($connection, $unionQueryBuilder, $parentId);
return $unionQueryBuilder
->union($firstPartQueryBuilder)
->addUnion($secondPartQueryBuilder, UnionType::DISTINCT)
->orderBy('uid', 'ASC')
->executeQuery()
->fetchAllAssociative();
}
private function getUnionPart1QueryBuilder(
Connection $connection,
QueryBuilder $unionQueryBuilder,
int $pageId,
): QueryBuilder {
$queryBuilder = $connection->createQueryBuilder();
// The union Expression Builder **must** be used on subqueries
$unionExpr = $unionQueryBuilder->expr();
$queryBuilder
// The column names of the first query are used
// The column count of both subqueries must be the same
// The data types must be compatible across columns of the queries
->select('title', 'subtitle')
->from('pages')
->where(
// The union Expression Builder **must** be used on subqueries
$unionExpr->eq(
'pages.pid',
// Named parameters **must** be created on the outermost (union) query builder
$unionQueryBuilder->createNamedParameter($pageId, Connection::PARAM_INT),
),
);
return $queryBuilder;
}
private function getUnionPart2QueryBuilder(
Connection $connection,
QueryBuilder $unionQueryBuilder,
int $pageId,
): QueryBuilder {
$queryBuilder = $connection->createQueryBuilder();
// The union Expression Builder **must** be used on subqueries
$unionExpr = $unionQueryBuilder->expr();
$queryBuilder
// The column count of both subqueries must be the same
->select('header', 'subheader')
->from('tt_content')
->where(
$unionExpr->eq(
'tt_content.pid',
// Named parameters **must** be created on the outermost (union) query builder
$unionQueryBuilder->createNamedParameter($pageId, Connection::PARAM_INT),
),
);
return $queryBuilder;
}
}
- Line 18
- All query parts must share the same connection.
- Line 19
- The outer most QueryBuilder is responsible for the union, it must be used to create named parameters and build expressions within the sub queries.
- Line 22-23
- We therefore pass the central QueryBuilder responsible for the
UNIONto all subqueries. Same with the ExpressionBuilder. - Line 25-30
- We start building the
unionon the first sub query, then add the second sub query using() addUnion () - Line 41
- Only use the ExpressionBuilder of the sql:
UNIONwithin the subqueries. - Line 50
- Named parameters must also be called on the outer most union query builder.
The Default Restrictions are applied to each subquery automatically.
setMaxResults() and setFirstResult()
Add
LIMIT to restrict the number of records and
OFFSET for
pagination of query parts. Both methods should be called only once per
statement:
// SELECT * FROM `sys_language` LIMIT 2 OFFSET 4
$queryBuilder = $this->connectionPool->getQueryBuilderForTable('sys_language');
$queryBuilder
->select('*')
->from('sys_language')
->setMaxResults(2)
->setFirstResult(4)
->executeQuery();
Read how to correctly instantiate a query builder with the connection pool.
Remarks:
- It is allowed to call
->setwithout callingMax Results () ->set.First Result () - It is possible to call
->setwithout callingFirst Result () set: This is equivalent to "Fetch everything, but leave out the first n records". Internally,Max Results () LIMITwill be added by Doctrine DBAL and set to a very high value. ->setcan be used to retrieve all results. If an unlimited result set is needed, and no reset of previous instructions is required, this method call should best be omitted for best compatibility.Max Results (null)
Changed in version 13.0
Starting with TYPO3 13 null instead of argument 0 (integer)
must be used in
->set to return
the complete result set without any
LIMIT.
add()
Changed in version 13.0
With the upgrade to Doctrine DBAL version 4 this method has been removed.
Migration: use the direct methods instead:
| Before | After |
|---|---|
->add |
->select |
->add |
->where |
->add |
->having |
->add |
->order |
->add |
->group |
getSQL()
The
->get method returns the created query statement as string. It
is incredibly useful during development to verify that the final statement is
executed exactly as a developer expects:
$queryBuilder = $this->connectionPool->getQueryBuilderForTable('sys_language');
$queryBuilder
->select('*')
->from('sys_language');
debug($queryBuilder->getSQL());
$result = $queryBuilder->executeQuery();
Read how to correctly instantiate a query builder with the connection pool.
Remarks:
- This is debugging code. Take proper actions to ensure that these calls do not end up in production!
- The method is usually called directly before
->executeorQuery () ->executeto output the final statement.Statement () - Casting a query builder object to
(string)has the same effect as calling->get, but the explicit call using the method should be preferred to simplify a search operation for this kind of debugging statements.SQL () - The method is a simple way to see what restrictions the RestrictionBuilder has added.
- Doctrine DBAL always creates prepared statements: Each value added via
createNamedParameter
creates a placeholder that is later replaced when the real query is
triggered via
->executeorQuery () ->execute.Statement () ->getdoes not show these values, instead it displays the placeholder names, usually with a string likeSQL () :dc. There is no simple solution to show the fully replaced query within the framework, but you can use getParameters to see the array of parameters used to replace these placeholders within the query. On the frontend, the queries and parameters are available in the admin panel.Value1
getParameters()
The
->get method returns the values for the placeholders of
the prepared statement in an array. This is incredibly useful during development
to verify that the final statement is executed as a developer expects:
$queryBuilder = $this->connectionPool->getQueryBuilderForTable('sys_language');
$queryBuilder
->select('*')
->from('sys_language');
debug($queryBuilder->getParameters());
$statement = $queryBuilder->executeQuery();
Read how to correctly instantiate a query builder with the connection pool.
Remarks:
- This is debugging code. Take proper actions to ensure that these calls do not end up in production!
- The method is usually called directly before
->executeorQuery () ->executeto output the final statement.Statement () - Doctrine DBAL always creates prepared statements: Each value added via
createNamedParameter
creates a placeholder that is later replaced when the real query is
triggered via
->executeorQuery () ->execute.Statement () ->getdoes not show the statement or the placeholders, instead the values are displayed, usually an array using keys likeParameters () :dc. There is no simple solution to show the fully replaced query within the framework, but you can use getSql to see the string with placeholders, which is used as a prepared statement.Value1
executeQuery() and executeStatement()
Changed in version 13.0
The
->execute method has been removed. Use
->executereturning aQuery () \Doctrine\instead of aDBAL\ Result \Doctrine\(like theDBAL\ Statement ->executemethod returned) and() ->executereturning the number of affected rows.Statement ()
executeQuery()
This method compiles and fires the final query statement. This is usually the
last call on a query builder object. It can be called for
SELECT and
COUNT queries.
On success, it returns a result object of type
\Doctrine\
representing the result set. The
Result object can then be used by
fetch,
fetch and
fetch.
execute returns a
\Doctrine\ and not a
\Doctrine\ anymore.
Note
It is not possible to rebind placeholder values on the result and execute
another query, as was sometimes done with the
Statement returned by
execute.
If the query fails for some reason (for instance, if the database connection
was lost or if the query contains a syntax error), an
\Doctrine\ is thrown. It is usually bad habit to catch and
suppress this exception, as it indicates a runtime error a program error. Both
should bubble up. For more information on proper exception handling, see the
coding guidelines.
executeStatement()
The
execute method can be used for
INSERT,
UPDATE and
DELETE statements. It returns the number of affected
rows as an integer.
expr()
This method returns an instance of the ExpressionBuilder. It is used to create complex
WHERE query
parts and
JOIN expressions:
// use TYPO3\CMS\Core\Database\Connection;
// SELECT `uid` FROM `tt_content` WHERE (`uid` > 42)
$queryBuilder = $this->connectionPool->getQueryBuilderForTable('tt_content');
$queryBuilder
->select('uid')
->from('tt_content')
->where(
$queryBuilder->expr()->gt(
'uid',
$queryBuilder->createNamedParameter(42, Connection::PARAM_INT)
)
)
->executeQuery();
Read how to correctly instantiate a query builder with the connection pool. See available parameter types.
Remarks:
- This object is stateless and can be called and worked on as often as needed. However, it is bound to the specific connection for which a statement is created and therefore only available through the query builder, which is specific to a connection.
- Never reuse the ExpressionBuilder,
especially not between multiple query builder objects, but always get an
instance of the expression builder by calling
->expr.()
createNamedParameter()
Changed in version 13.0
Doctrine DBAL v4 dropped the support for using the
\PDO::
constants in favor of the enum types. Be aware of this and use
\TYPO3\, which can already be
used in TYPO3 v12 and v11.
This method creates a placeholder for a field value of a prepared statement. Always use this when dealing with user input in expressions to protect the statement from SQL injections:
// SELECT * FROM `tt_content` WHERE (`bodytext` = 'kl\'aus')
$searchWord = "kl'aus"; // $searchWord retrieved from the PSR-7 request
$queryBuilder = $this->connectionPool->getQueryBuilderForTable('tt_content');
$queryBuilder->getRestrictions()->removeAll();
$queryBuilder
->select('uid')
->from('tt_content')
->where(
$queryBuilder->expr()->eq(
'bodytext',
$queryBuilder->createNamedParameter($searchWord, Connection::PARAM_STR)
)
)
->executeQuery();
Read how to correctly instantiate a query builder with the connection pool. See available parameter types.
The above example shows the importance of using
->create:
The search word kl'aus is "tainted" and would break the query if not
channeled through
->create, which quotes the backtick and
makes the value SQL injection-safe.
Not convinced? Suppose the code would look like this:
// NEVER EVER DO THIS!
$_POST['searchword'] = "'foo' UNION SELECT username FROM be_users";
$searchWord = $request->getParsedBody()['searchword']);
$queryBuilder = $this->connectionPool->getQueryBuilderForTable('tt_content');
$queryBuilder->getRestrictions()->removeAll();
this fails with syntax error to prevent copy and paste
$queryBuilder
->select('uid')
->from('tt_content')
->where(
// MASSIVE SECURITY ISSUE DEMONSTRATED HERE
// USE ->createNamedParameter() ON $searchWord!
$queryBuilder->expr()->eq('bodytext', $searchWord)
);
Read how to correctly instantiate a query builder with the connection pool.
Mind the missing
->create method call in the
->eq expression for a given value! This code would happily execute
the statement
SELECT uid FROM `tt_content` WHERE `bodytext` = 'foo' UNION SELECT username FROM be_users;
returning a list of backend user names!
Note
->set automatically converts the second mandatory parameter into
a named parameter of a prepared statement. If the second parameter is
wrapped in a
->create call, this will result in an
error during execution. This behaviour can be disabled by passing
false as third parameter to
->set.
More examples
Use integer, integer array:
// use TYPO3\CMS\Core\Database\Connection;
// SELECT * FROM `tt_content`
// WHERE `bodytext` = 'kl\'aus'
// AND sys_language_uid = 0
// AND pid in (2, 42,13333)
$searchWord = "kl'aus"; // $searchWord retrieved from the PSR-7 request
$queryBuilder = $this->connectionPool->getQueryBuilderForTable('tt_content');
$queryBuilder->getRestrictions()->removeAll();
$queryBuilder
->select('uid')
->from('tt_content')
->where(
$queryBuilder->expr()->eq(
'bodytext',
$queryBuilder->createNamedParameter($searchWord)
),
$queryBuilder->expr()->eq(
'sys_language_uid',
$queryBuilder->createNamedParameter($language, Connection::PARAM_INT)
),
$queryBuilder->expr()->in(
'pid',
$queryBuilder->createNamedParameter($pageIds, Connection::PARAM_INT_ARRAY)
)
)
->executeQuery();
Read how to correctly instantiate a query builder with the connection pool. See available parameter types.
Rules
- Always use
->createfor any input, no matter where it comes from.Named Parameter () - The second argument of
->expris always either a call to() ->createorNamed Parameter () ->quote.Identifier () - The second argument of
->createspecifies the type of input. For string, this can be omitted, but it is good practice to addNamed Parameter () \TYPO3\for integers or similar for other field types. This is not strict rule currently, but if you follow it you will have fewer headaches in the future, especially with DBMSes that are not as relaxed as MySQL when it comes to field types. TheCMS\ Core\ Database\ Connection:: PARAM_ INT Connectionconstants can be used for simple types likebool,string,null,lobandinteger. Additionally, the two constantsConnection::andPARAM_ INT_ ARRAY Connection::can be used when handling an array of strings or integers, for instance in anPARAM_ STR_ ARRAY INexpression.() - Keep the
->createmethod as close to the expression as possible. Do not structure your code in a way that it quotes something first and only later stuffs the already prepared names into the expression. HavingNamed Parameter () ->createdirectly within the created expression, is much less error-prone and easier to review. This is a general rule: Sanitizing input must be done as close as possible to the "sink" where a value is passed to a lower part of the framework. This paradigm should also be followed for other quote operations likeNamed Parameter () htmlspecialcharsor() General. Sanitization should be obvious directly at the very place where it is important:Utility:: quote JSvalue ()
// use TYPO3\CMS\Core\Database\Connection;
// DO
$queryBuilder = $this->connectionPool->getQueryBuilderForTable('tt_content');
$queryBuilder->getRestrictions()->removeAll();
$queryBuilder
->select('uid')
->from('tt_content')
->where(
$queryBuilder->expr()->eq(
'bodytext',
$queryBuilder->createNamedParameter($searchWord, Connection::PARAM_STR)
)
)
// DON'T DO, this is much harder to track:
$queryBuilder = $this->connectionPool->getQueryBuilderForTable('tt_content');
$myValue = $queryBuilder->createNamedParameter($searchWord);
// Imagine much more code here
$queryBuilder->getRestrictions()->removeAll();
$queryBuilder
->select('uid')
->from('tt_content')
->where(
$queryBuilder->expr()->eq('bodytext', $myValue)
)
Read how to correctly instantiate a query builder with the connection pool. See available parameter types.
quoteIdentifier() and quoteIdentifiers()
->quote must be used when not a value but a field name is
handled. The quoting is different in those cases and typically ends up with
backticks ` instead of ticks ':
// SELECT `uid` FROM `tt_content` WHERE (`header` = `bodytext`)
// Return list of rows where header and bodytext values are identical
$queryBuilder = $this->connectionPool->getQueryBuilderForTable('tt_content');
$queryBuilder
->select('uid')
->from('tt_content')
->where(
$queryBuilder->expr()->eq(
'header',
$queryBuilder->quoteIdentifier('bodytext')
)
);
Read how to correctly instantiate a query builder with the connection pool.
The method quotes single field names or combinations of table names or table aliases with field names:
// Single field name: `bodytext`
->quoteIdentifier('bodytext');
// Table name and field name: `tt_content`.`bodytext`
->quoteIdentifier('tt_content.bodytext')
// Table alias and field name: `foo`.`bodytext`
->from('tt_content', 'foo')->quoteIdentifier('foo.bodytext')
Remarks:
- Similar to ->createNamedParameter() this method is crucial to prevent SQL injections. The same rules apply here.
- The ->set() method for
UPDATEstatements expects its second argument to be a field value by default, and quotes it internally using->create. If a field should be set to the value of another field, this quoting can be turned off and an explicit call toNamed Parameter () ->quotemust be added.Identifier () - Internally,
->quoteis automatically called on all method arguments that must be a field name. For instance,Identifier () ->quoteis called for all arguments of ->select().Identifier () ->quote(mind the plural) can be used to quote multiple field names at once. While that method is "public" and thus exposed as an API method, this is mostly useful internally only.Identifiers ()
escapeLikeWildcards()
Helper method to quote % characters within a search string. This is helpful in
->like and
->not expressions:
// use TYPO3\CMS\Core\Database\Connection;
// SELECT `uid` FROM `tt_content` WHERE (`bodytext` LIKE '%kl\\%aus%')
$searchWord = 'kl%aus';
$queryBuilder = $this->connectionPool->getQueryBuilderForTable('tt_content');
$queryBuilder
->select('uid')
->from('tt_content')
->where(
$queryBuilder->expr()->like(
'bodytext',
$queryBuilder->createNamedParameter('%' . $queryBuilder->escapeLikeWildcards($searchWord) . '%', Connection::PARAM_STR)
)
);
Read how to correctly instantiate a query builder with the connection pool. See available parameter types.
Warning
Even when using
->escape the value must be
encapsulated again in a
->create call. Only calling
->escape does not make the value SQL injection
safe!
getRestrictions(), setRestrictions(), resetRestrictions()
API methods to deal with the RestrictionBuilder.