DEPRECATION WARNING
This documentation is not using the current rendering mechanism and is probably outdated. The extension maintainer should switch to the new system. Details on how to use the rendering mechanism can be found here.
EXT: Readable name of your extension¶
Author: | Kasper Skårhøj |
---|---|
Created: | 2002-11-01T00:32:00 |
Changed by: | w x |
Changed: | 2018-06-27T18:29:09.950000000 |
Classification: | extensionkey |
Description: | The keywords help with categorizing and tagging of the manuals. You can combine two or more keywords and add additional keywords yourself. Please use at least one keyword from both lists. If your manual is NOT in english, see next tab “language” —- forEditors (use this for editors / german “Redakteure”) forAdmins (use this for Administrators) forDevelopers (use this for Developers) forBeginners (manuals covering TYPO3 basics) forIntermediates (manuals going into more depth) forAdvanced (covering the most advanced TYPO3 topics) see more: http://wiki.typo3.org/doc_template#tags —- |
Keywords: | keywords comma-separated |
Author: | Author Name |
Email: | your@email.com |
Info 4: | |
Language: | en |
Dump SQL table/query to CSV -w_query2csvDump SQL table/query to CSV
Dump SQL table/query to CSV¶
Extension Key: w_query2csv
Language: en
Keywords: csv, sql, dump, export, table, query, spreadsheet, excel, calc
Copyright 2009-2018, wolo.pl ‘.’ studio, <wolo.wolski@gmail.com>
https://github.com/w010/w_query2csv/
This document is published under the Open Content License
available from http://www.opencontent.org/opl.shtml
The content of this document is related to TYPO3
- a GNU/GPL CMS/Framework available from www.typo3.org
Table of Contents¶
`Introduction 3 <#__RefHeading__758_431020924>`_
`How to use it? 4 <#__RefHeading__766_431020924>`_
`Administration / configuration 6 <#__RefHeading__772_431020924>`_
Reference / options description 6
`Security 9 <#__RefHeading__1013_431020924>`_
`FAQ 10 <#__RefHeading__782_431020924>`_
`Migrating 11 <#__RefHeading__1034_431020924>`_
`Known problems 12 <#__RefHeading__784_431020924>`_
`To-Do list 13 <#__RefHeading__786_431020924>`_
`ChangeLog 14 <#__RefHeading__788_431020924>`_
Introduction¶
What is this?¶
This is extension for quick export SQL table or query/ies to CSV file/s. You may configure number of files which will be available, for every file using their own settings, select different table, fields which will be exported, standard parts of MySQL query, file charset, csv fields separator and more. All that is fast and easy, using included example TypoScript.
You may also parse selected fields with own methods, like tstamp to human-readable date.
Please see “Security” section before use¶
For what do I need it?¶
If you have data set in db, eg. some orders or contest answers, that must be quick sent to someone by email, or periodically downloaded by someone, or something like that, this extension is very useful.
Just make new page, insert plugin and set up the file, that will be downloaded by other or specified users, which you may specify by standard TYPO3 access settings.
It’s simple and easy to configure, additionally you may process selected fields using some functions, eg. convert timestamps to human- readable strings.
Want something more functional? It is a good base to develop something bigger.
Screenshots¶
How to use it?¶
Basic¶
Simply insert plugin on new page named eg. ‘CSV export’ and configure input and output using TypoScript.
A Backend Section type page is good idea, you might control download access for backend non-admin users.
To download file, you have to access url using ?f=[configuration_key], as here:
http://example.com/csv_export.html ?f=somestuff_orders
Note, that this plugin is not basing on standard TYPO3 page type handling, but is changing headers for whole page containing it, so do not insert it on standard frontend pages with normal content.
But it’s possible to make CSV output of any page uid using page type, if you use a snippet from Configuration/TypoScript/Setup/setup.ts
Caution where do you insert this plugin and what is exported!
Using in own plugins¶
One time I needed to display link to csv in my other plugin context. This needs to make query2csv object, prepare and pass config to it, and call when requested.
This can be done like:
Example:¶
$content .= '<a href="'.GeneralUtility::linkThisUrl($_SERVER['REQUEST_URI'], array('action' => 'getfile', 'f' => 'my_file')) . '">download file</a>';
if ($_GET['action'] == 'getfile') {
require_once(ExtensionManagementUtility::extPath('w_query2csv').'Classes/Plugins/Export.php');
$conf = [
'debug_allowed' => 0,
'files.' => [
'my_file.' => [
'input.' => [
'table' => 'tx_myext_table',
'fields' => 'uid, crdate, name, value',
'where' => 'AND type = 2',
'order' => 'uid DESC',
'limit' => 100,
],
'output.' => [
'filename' => 'output.csv',
'process_fields.' => [
'crdate' => 'WoloPl\WQuery2csv\Process->parseDate',
]]]]];
$Q2csv = GeneralUtility::makeInstance('tx_wquery2csv_export');
$Q2csv->main('', $conf);
// note, that the script now throw output and exits!
}
Administration / configuration¶
For selected page containing plugin, create new TypoScript template and edit Setup field.
((generated))¶
Example config:¶
plugin.tx_wquery2csv_export {
files {
my_file {
input {
table = tx_somestuff_orders
fields = tstamp,name,email,phone,stuff_uid
where = category = 2
group =
order = tstamp DESC
limit =
enable_columns = 1
}
output {
filename = somestuff_orders.csv
separator = ,
encoding =
process_fields {
tstamp = WoloPl\WQuery2csv\Process->parseDate
}
}
}
.... (some other keys with input and output options to access file using ?f=some_other_key)
}
debug_allowed = 1
}
Shortest working example:¶
When you take a look at reference below, you’ll see, that only db table is really required in configuration, so shortest config would look like this:
plugin.tx_wquery2csv_export.files.my_file.input.table = tx_sometable
Explaining:¶
file key “ my_file ” has set input table to “ tx_sometable ”
so:
the file will be accessed like: [download_csv.html]?f= my_file
and:
prompted do download as my_file.csv
Reference / options description¶
((generated))¶
plugin.tx_wquery2csv_export¶
Property
files
Data type
List of TypoScript properties
Description
(Required)
Set of file keys to separate configs for misc files. Keynames are up to you. if you want only one file, the key may be whatever.
If you don’t set output.filename option, the filename will be like your key named here.
Every key must have configured at least “ input ” section. “ output ” is not required, but has some useful options.
Default
Property
debug_allowed
Data type
Boolean
Description
Allow to use &debug=1 url param.
Default
0
Property
default_config_if_missed
Data type
Boolean
Description
Load ‘_default’ ts key config when config not found.
Default
0
file options¶
Property
input
Data type
List of TypoScript properties
->input
Description
(Required)
Options for input data to render file from
@see table below
Default
Property
output
Data type
List of TypoScript properties
->output
Description
Miscellaneous options for output file
@see table below
Default
Property
disable
Data type
Boolean
Description
Use to quick disable access to configured file without deleting or commenting ts file config setup
Default
0
input¶
Property
table
Data type
List of TypoScript properties
Description
(Required)
Options for input data to render file from
Default
Property
fields
Data type
List of TypoScript properties
Description
Miscellaneous options for output file
Default
*
Property
where, group, order, limit
Data type
String
Description
Standard parts of database query
Default
Property
where_wrap
Data type
TypoScript object
Description
Builds where clause using stdWrap and replaces ‘where’ if set. Use:
where_wrap = TEXT
where_wrap.value = pid = {page:uid}
where_wrap.stdWrap.insertData = 1
Default
Property
enableFields
Data type
Boolean
Description
Use standard enableFields call to filter unavailable fields. Works only for TCA configured tables
DEFAULT IS ENABLED (so if you has given empty csv, check that selected table has these fields and set this option to 0 if not)
Default
1
Property
default_enableColumns
Data type
Description
deleted=0 AND hidden=0 in WHERE clause.
Default
output¶
Property
filename
Data type
String
Description
Output filename
Default
same as config file key with csv extension added
Property
separator
Data type
String
Description
CSV separator used in file
Default
,
Property
charset
Data type
String
Description
Set alternative charset encoding
Default
UTF-8
Property
hsc
Data type
Boolean
Description
Htmlspecialchars every value
Default
0
Property
no_header_row
Data type
Boolean
Description
Don’t make first line header with fieldnames
Default
0
Property
process_fields
Data type
List of TypoScript properties
Description
Set of TS properties like:
field = process_method
field.[setting] = …
Parses selected db fields with plugin class method named as value.
Example:
tstamp = WoloPl\WQuery2csv\Process->parseDate
tstamp.format = d.m.Y H:i
Built-in processors reference:
->parseDate
Parses value with date() function. Config:
.format = [string]
->valueMap
Maps input values, like some uid, to output values, like nice labels:
.map { [array]
source value = Export Value
[in val = out val pairs]
}
->staticValue
Replaces original value with given – may be handy in some cases:
.value = [string]
->unserialize
.delimiter = [string] - may be anything, also you can use special keywords: -LINEBREAK- and -SPACE-. defaults to linebreak
.lineBreakType = [string] - if use linebreak, may be configured to use CR, LF or CRLF. defaults to LF
->tableLabelsFromRecordsCommalist
.table = [string] - table name to read referenced items from
.field = [string] - use value from this field
.delimiter = [string] - may be anything, also you can use special keywords: -LINEBREAK- and -SPACE-. defaults to linebreak
.lineBreakType = [string] - may be CR, LF or CRLF. defaults to LF
You can easily make own processors the same way
Default
Property
add_fields
Data type
String - commalist
Description
Create additional columns in output, that are not present in database. Set values for them in process_fields.
Default
Property
remove_fields
Data type
String - commalist
Description
Remove columns from output (values of which might be used for some processing, but they are not needed in csv)
Default
The easiest way to quick configure is to just copy example config from above and edit options to your own.
Security¶
PLEASE REMEMBER¶
This is a low-level database exporter, which was intended to use by admins to easy download customized csv with product orders. Such things are always potentially dangerous if used in wrong way. Downloading full database tables by people who are not permitted may be a disaster - passwords leak, session hijack, etc… - so better check twice where do you put this plugin and what have you configured there, to not allow downloading any sensitive data by mistake!
I recommend always embeding it on pages with BE-user or admin access only, unless you are sure what are you doing.
FAQ¶
Q: How to add others functions to process fields values?
A: Just make a class like w_query2csv/Classes/Process.php and put it in your extension Classes directory. Define valid namespace and use as described above in process_fields section (this is standard TYPO3’s userfunc reference, instantiated by GeneralUtility).
In this class you add public method with the name you chose, which expects 2 parameters: $params, $pObj.
$params - array of keys:value (string)conf (array) – contains config for field, like format or maprow (array) – full row datafieldname (string)$pObj - reference to Core object, which sometimes may be useful.
Q: My output file is empty!
A: - try to set debug_allowed = 1 in config and access file with &debug=1 to check if the config is passed properly.
- try to set “fields” to *, comment other parts of db query
- check another table
- set “enableFields” to 0 (not just delete line!), maybe the table is not loaded in TCA
Q: What if BE-user has template edit privileges and exports something that he is not allowed?
Why I cannot export my fe_users?
A: Some tables would never be accessed even by other backend users. To prevent situation when a user configure plugin to see ie. users passwords, or allow a table which is originally blocked, set selected tables as comma separated list on “not_allowed_tables”, in LocalConfiguration or AdditionalConfiguration
$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['w_query2csv'] = [
'not_allowed_tables' => 'be_users,be_groups,be_sessions,fe_users,fe_groups,fe_sessions,fe_session_data',
];
Migrating¶
Migrate from version 0.1¶
- typoscript setup key is now: plugin.tx_wquery2csv_export instead of plugin.tx_wquery2csv_pi1
- so the plugin content element embeded on page must be selected again and setup must be adjusted
- process_fields now expects full userfunc reference
- so, _process_parseDate option is now: WoloPlWQuery2csvProcess->parseDate
- process_fields_user is now removed, use process_fields instead, just like the rest
- output.where now need to be full, not starting from “AND”
Known problems¶
If you see any, let me know.
To-Do list¶
If you would like to see some functionality in my extension, let me know.
ChangeLog¶
0.1.1
First release with full documentation.
0.1.5
Last built query is saved and available on debug,
Quotes in input are escaped to csv-compatible double quotes (was messing whole csv)
Additional tables added to not_allowed_tables
0.2.0
Code reworked and compatible with TYPO3 version 7.x and 8.x
Userfunc field processing
Many minor improvements – some of them breaks compatibility. Please refer to Migration section, if updating
0.3.0
Charset converting - iconv changed to mbstring (thanks to Henri Nathanson for suggestion)
Inactive records filtering changed to native - now use enableFields = 1 for this. use default_enableColumns only when table isn’t in TCA
Option add_fields fixed - works again
Debugging now works automatically when TYPO3_CONTEXT == Development, no need to set debug_allowed = 1
Process->unserialize now can use custom delimiter
New processors: ->tableLabelsFromRecordsCommalist, ->tableLabelsFromMmRelations
14