DEPRECATION WARNING

This documentation is not using the current rendering mechanism and is probably outdated. The extension maintainer should switch to the new system. Details on how to use the rendering mechanism can be found here.

EXT: Readable name of your extension

Author:Kasper Skårhøj
Created:2002-11-01T00:32:00
Changed by:w x
Changed:2018-06-27T18:29:09.950000000
Classification:extensionkey
Description:The keywords help with categorizing and tagging of the manuals. You can combine two or more keywords and add additional keywords yourself. Please use at least one keyword from both lists. If your manual is NOT in english, see next tab “language” —- forEditors (use this for editors / german “Redakteure”) forAdmins (use this for Administrators) forDevelopers (use this for Developers) forBeginners (manuals covering TYPO3 basics) forIntermediates (manuals going into more depth) forAdvanced (covering the most advanced TYPO3 topics) see more: http://wiki.typo3.org/doc_template#tags —-
Keywords:keywords comma-separated
Author:Author Name
Email:your@email.com
Info 4:
Language:en

img-1 img-2 Dump SQL table/query to CSV -w_query2csvDump SQL table/query to CSV

Dump SQL table/query to CSV

Extension Key: w_query2csv

Language: en

Keywords: csv, sql, dump, export, table, query, spreadsheet, excel, calc

Copyright 2009-2018, wolo.pl ‘.’ studio, <wolo.wolski@gmail.com>

https://github.com/w010/w_query2csv/

This document is published under the Open Content License

available from http://www.opencontent.org/opl.shtml

The content of this document is related to TYPO3

- a GNU/GPL CMS/Framework available from www.typo3.org

Table of Contents

Dump SQL table/query to CSV 1

`Introduction 3 <#__RefHeading__758_431020924>`_

What is this? 3

For what do I need it? 3

Screenshots 3

`How to use it? 4 <#__RefHeading__766_431020924>`_

Basic 4

Using in own plugins 5

`Administration / configuration 6 <#__RefHeading__772_431020924>`_

Reference / options description 6

`Security 9 <#__RefHeading__1013_431020924>`_

PLEASE REMEMBER 9

`FAQ 10 <#__RefHeading__782_431020924>`_

`Migrating 11 <#__RefHeading__1034_431020924>`_

Migrate from version 0.1 11

`Known problems 12 <#__RefHeading__784_431020924>`_

`To-Do list 13 <#__RefHeading__786_431020924>`_

`ChangeLog 14 <#__RefHeading__788_431020924>`_

Introduction

What is this?

This is extension for quick export SQL table or query/ies to CSV file/s. You may configure number of files which will be available, for every file using their own settings, select different table, fields which will be exported, standard parts of MySQL query, file charset, csv fields separator and more. All that is fast and easy, using included example TypoScript.

You may also parse selected fields with own methods, like tstamp to human-readable date.

Please see “Security” section before use

For what do I need it?

If you have data set in db, eg. some orders or contest answers, that must be quick sent to someone by email, or periodically downloaded by someone, or something like that, this extension is very useful.

Just make new page, insert plugin and set up the file, that will be downloaded by other or specified users, which you may specify by standard TYPO3 access settings.

It’s simple and easy to configure, additionally you may process selected fields using some functions, eg. convert timestamps to human- readable strings.

Want something more functional? It is a good base to develop something bigger.

Screenshots

img-3

How to use it?

Basic

Simply insert plugin on new page named eg. ‘CSV export’ and configure input and output using TypoScript.

A Backend Section type page is good idea, you might control download access for backend non-admin users.

To download file, you have to access url using ?f=[configuration_key], as here:

http://example.com/csv_export.html ?f=somestuff_orders

Note, that this plugin is not basing on standard TYPO3 page type handling, but is changing headers for whole page containing it, so do not insert it on standard frontend pages with normal content.

But it’s possible to make CSV output of any page uid using page type, if you use a snippet from Configuration/TypoScript/Setup/setup.ts

Caution where do you insert this plugin and what is exported!

img-4

Using in own plugins

One time I needed to display link to csv in my other plugin context. This needs to make query2csv object, prepare and pass config to it, and call when requested.

This can be done like:

Example:
$content .= '<a href="'.GeneralUtility::linkThisUrl($_SERVER['REQUEST_URI'], array('action' => 'getfile', 'f' => 'my_file')) . '">download file</a>';


if ($_GET['action'] == 'getfile')       {
        require_once(ExtensionManagementUtility::extPath('w_query2csv').'Classes/Plugins/Export.php');

        $conf = [
                'debug_allowed' => 0,
                'files.' => [
                        'my_file.' => [
                                'input.' => [
                                        'table' => 'tx_myext_table',
                                        'fields' => 'uid, crdate, name, value',
                                        'where' => 'AND type = 2',
                                        'order' => 'uid DESC',
                                        'limit' => 100,
                                ],
                                'output.' => [
                                        'filename' => 'output.csv',
                                        'process_fields.' => [
                                                'crdate' => 'WoloPl\WQuery2csv\Process->parseDate',
        ]]]]];


        $Q2csv = GeneralUtility::makeInstance('tx_wquery2csv_export');

        $Q2csv->main('', $conf);
        // note, that the script now throw output and exits!
}

Administration / configuration

For selected page containing plugin, create new TypoScript template and edit Setup field.

((generated))

Example config:
plugin.tx_wquery2csv_export  {
    files   {
        my_file {
            input {
              table = tx_somestuff_orders
              fields = tstamp,name,email,phone,stuff_uid
              where = category = 2
              group =
              order = tstamp DESC
              limit =
              enable_columns = 1
            }
            output {
              filename = somestuff_orders.csv
              separator = ,
              encoding =
              process_fields {
                tstamp = WoloPl\WQuery2csv\Process->parseDate
              }
            }
        }

        .... (some other keys with input and output options to access file using ?f=some_other_key)
    }

    debug_allowed = 1
}
Shortest working example:

When you take a look at reference below, you’ll see, that only db table is really required in configuration, so shortest config would look like this:

plugin.tx_wquery2csv_export.files.my_file.input.table = tx_sometable
Explaining:

file key “ my_file ” has set input table to “ tx_sometable

so:

the file will be accessed like: [download_csv.html]?f= my_file

and:

prompted do download as my_file.csv

Reference / options description

((generated))
plugin.tx_wquery2csv_export
files

Property

files

Data type

List of TypoScript properties

Description

(Required)

Set of file keys to separate configs for misc files. Keynames are up to you. if you want only one file, the key may be whatever.

If you don’t set output.filename option, the filename will be like your key named here.

Every key must have configured at least “ input ” section. “ output ” is not required, but has some useful options.

Default

debug_allowed

Property

debug_allowed

Data type

Boolean

Description

Allow to use &debug=1 url param.

Default

0

default_config_if_missed

Property

default_config_if_missed

Data type

Boolean

Description

Load ‘_default’ ts key config when config not found.

Default

0

file options
input

Property

input

Data type

List of TypoScript properties

->input

Description

(Required)

Options for input data to render file from

@see table below

Default

output

Property

output

Data type

List of TypoScript properties

->output

Description

Miscellaneous options for output file

@see table below

Default

disable

Property

disable

Data type

Boolean

Description

Use to quick disable access to configured file without deleting or commenting ts file config setup

Default

0

input
table

Property

table

Data type

List of TypoScript properties

Description

(Required)

Options for input data to render file from

Default

fields

Property

fields

Data type

List of TypoScript properties

Description

Miscellaneous options for output file

Default

*

where, group, order, limit

Property

where, group, order, limit

Data type

String

Description

Standard parts of database query

Default

where_wrap

Property

where_wrap

Data type

TypoScript object

Description

Builds where clause using stdWrap and replaces ‘where’ if set. Use:

where_wrap = TEXT

where_wrap.value = pid = {page:uid}

where_wrap.stdWrap.insertData = 1

Default

enableFields

Property

enableFields

Data type

Boolean

Description

Use standard enableFields call to filter unavailable fields. Works only for TCA configured tables

DEFAULT IS ENABLED (so if you has given empty csv, check that selected table has these fields and set this option to 0 if not)

Default

1

default_enableColumns

Property

default_enableColumns

Data type

Description

deleted=0 AND hidden=0 in WHERE clause.

Default

output
filename

Property

filename

Data type

String

Description

Output filename

Default

same as config file key with csv extension added

separator

Property

separator

Data type

String

Description

CSV separator used in file

Default

,

charset

Property

charset

Data type

String

Description

Set alternative charset encoding

Default

UTF-8

hsc

Property

hsc

Data type

Boolean

Description

Htmlspecialchars every value

Default

0

no_header_row

Property

no_header_row

Data type

Boolean

Description

Don’t make first line header with fieldnames

Default

0

process_fields

Property

process_fields

Data type

List of TypoScript properties

Description

Set of TS properties like:

field = process_method

field.[setting] = …

Parses selected db fields with plugin class method named as value.

Example:

tstamp = WoloPl\WQuery2csv\Process->parseDate
tstamp.format = d.m.Y H:i

Built-in processors reference:

->parseDate

Parses value with date() function. Config:

.format = [string]

->valueMap

Maps input values, like some uid, to output values, like nice labels:

.map {    [array]
    source value = Export Value
    [in val = out val pairs]
}

->staticValue

Replaces original value with given – may be handy in some cases:

.value = [string]

->unserialize

.delimiter = [string] - may be anything, also you can use special keywords: -LINEBREAK- and -SPACE-. defaults to linebreak

.lineBreakType = [string] - if use linebreak, may be configured to use CR, LF or CRLF. defaults to LF

->tableLabelsFromRecordsCommalist

.table = [string] - table name to read referenced items from

.field = [string] - use value from this field

.delimiter = [string] - may be anything, also you can use special keywords: -LINEBREAK- and -SPACE-. defaults to linebreak

.lineBreakType = [string] - may be CR, LF or CRLF. defaults to LF

You can easily make own processors the same way

Default

add_fields

Property

add_fields

Data type

String - commalist

Description

Create additional columns in output, that are not present in database. Set values for them in process_fields.

Default

remove_fields

Property

remove_fields

Data type

String - commalist

Description

Remove columns from output (values of which might be used for some processing, but they are not needed in csv)

Default

The easiest way to quick configure is to just copy example config from above and edit options to your own.

Security

PLEASE REMEMBER

This is a low-level database exporter, which was intended to use by admins to easy download customized csv with product orders. Such things are always potentially dangerous if used in wrong way. Downloading full database tables by people who are not permitted may be a disaster - passwords leak, session hijack, etc… - so better check twice where do you put this plugin and what have you configured there, to not allow downloading any sensitive data by mistake!

I recommend always embeding it on pages with BE-user or admin access only, unless you are sure what are you doing.

FAQ

Q: How to add others functions to process fields values?

A: Just make a class like w_query2csv/Classes/Process.php and put it in your extension Classes directory. Define valid namespace and use as described above in process_fields section (this is standard TYPO3’s userfunc reference, instantiated by GeneralUtility).

In this class you add public method with the name you chose, which expects 2 parameters: $params, $pObj.

$params - array of keys:value (string)conf (array) – contains config for field, like format or maprow (array) – full row datafieldname (string)$pObj - reference to Core object, which sometimes may be useful.

Q: My output file is empty!

A: - try to set debug_allowed = 1 in config and access file with &debug=1 to check if the config is passed properly.

- try to set “fields” to *, comment other parts of db query

- check another table

- set “enableFields” to 0 (not just delete line!), maybe the table is not loaded in TCA

Q: What if BE-user has template edit privileges and exports something that he is not allowed?

Why I cannot export my fe_users?

A: Some tables would never be accessed even by other backend users. To prevent situation when a user configure plugin to see ie. users passwords, or allow a table which is originally blocked, set selected tables as comma separated list on “not_allowed_tables”, in LocalConfiguration or AdditionalConfiguration

$GLOBALS['TYPO3_CONF_VARS']['EXTCONF']['w_query2csv'] = [
    'not_allowed_tables' => 'be_users,be_groups,be_sessions,fe_users,fe_groups,fe_sessions,fe_session_data',
];

Migrating

Migrate from version 0.1

- typoscript setup key is now: plugin.tx_wquery2csv_export instead of plugin.tx_wquery2csv_pi1

- so the plugin content element embeded on page must be selected again and setup must be adjusted

- process_fields now expects full userfunc reference

- so, _process_parseDate option is now: WoloPlWQuery2csvProcess->parseDate

- process_fields_user is now removed, use process_fields instead, just like the rest

- output.where now need to be full, not starting from “AND”

Known problems

If you see any, let me know.

To-Do list

If you would like to see some functionality in my extension, let me know.

ChangeLog

  • 0.1.1

    First release with full documentation.

  • 0.1.5

    Last built query is saved and available on debug,

    Quotes in input are escaped to csv-compatible double quotes (was messing whole csv)

    Additional tables added to not_allowed_tables

  • 0.2.0

    Code reworked and compatible with TYPO3 version 7.x and 8.x

    Userfunc field processing

    Many minor improvements – some of them breaks compatibility. Please refer to Migration section, if updating

  • 0.3.0

    Charset converting - iconv changed to mbstring (thanks to Henri Nathanson for suggestion)

    Inactive records filtering changed to native - now use enableFields = 1 for this. use default_enableColumns only when table isn’t in TCA

    Option add_fields fixed - works again

    Debugging now works automatically when TYPO3_CONTEXT == Development, no need to set debug_allowed = 1

    Process->unserialize now can use custom delimiter

    New processors: ->tableLabelsFromRecordsCommalist, ->tableLabelsFromMmRelations

14