DEPRECATION WARNING

This documentation is not using the current rendering mechanism and is probably outdated. The extension maintainer should switch to the new system. Details on how to use the rendering mechanism can be found here.

Media: demo extension

Author:Kasper Skårhøj
Created:2002-11-01T00:32:00
Changed:2006-02-28T17:18:57
Author:René Fritz
Email:r.fritz@colorcube.de
Info 3:
Info 4:

Media: demo extension

Extension Key: dam_demo

Copyright 2003-2006, René Fritz, <r.fritz@colorcube.de>

This document is published under the Open Content License

available from http://www.opencontent.org/opl.shtml

The content of this document is related to TYPO3

- a GNU/GPL CMS/Framework available from www.typo3.com

Table of Contents

Media: demo extension 1

Introduction 1

Extending DAM 1

The DAM record data structure 4

Multilanguage meta data 6

Selections 6

DAM services 7

Debugging 8

Introduction

This extension is an example how to extend the TYPO3 Digital Asset Management System (DAM).

This extension will later include more documentation for developers. For now: use the source Luke .

Extending DAM

The DAM system is extendable and customizable in various ways. It follows a short description of each feature. Some of them are available as example in the dam_demo extension.

Modules

The DAM system can be extended in the backend with out modules. They will be inserted and are accessable with entries in the tab menu of the main modules.

Additional database fields

see: dam_demo/add_fields/

Custom fields to the tx\_dam table will not only extend the table but these fields will be used automatically by the DAM in different task like indexing presets or searching.

img-1 In ext_tables.php the additional fields can be configured to be shown in the indexing preset and batch processing forms.

   // add fields to index preset fields
$TCA['tx_dam']['txdamInterface']['index_fieldList'] .= ',tx_damdemo_info,tx_damdemo_customcategory';

img-2

Indexing rule

see: dam_damo/indexrule/

base class: tx_dam_indexRuleBase

Do you have special files with meta data in it which needs to be processed while indexing? This can be done with own indexing rules.

img-3

Services for indexing

see: dam_demo/sv1/

base class: t3lib_svbase

The indexing function of the DAM currently uses three service types: metaExtract , textExtract and textLang .

For more information on services see the services documentation.

Previewer

see: dam_demo/previewer/

base class: tx_dam_previewerProcBase

Previewer are used to display a preview like a thumbnail or render a small gadget to see or play a file. The example below shows a simple MP3 player.

In principle several previewer types are possible. Implemented is currently a type that will be displayed in record editing view.

img-4

Actions

see: dam_demo/action/

base class: tx_dam_actionBase

Actions are GUI elements like icons or buttons that trigger a command for an item like a file or a record. Modules that use actions can easily be extended by own extensions.

The file module for example use three kind of actions which can be seen below.

img-5

Selection tree

see: dam_demo/selection/

base class: tx_dam_selBrowseTree

Selection trees provide a navigation tree and in the same time a selection class to create a SQL query from it's passed parameters. This is some kind of a misconception but for historical reason this will not be changed soon.

img-6

Selection class

see: dam/components/class.tx_dam_selectionStringSearch.php

base class: tx_dam_selProcBase

img-3 Selections are used to create SQL query parts depending on parameters. Pure selection classes have no GUI – in opposition to Selection trees.

The DAM record data structure

The DAM uses five database tables to store its data.

tx_dam DAM recordstx_dam_cat DAM category recordstx_dam_types_avail DAM file type recordstx_dam_mm_cat database relations between tx_dam and tx_dam_cattx_dam_mm_ref database relations between tx_dam and tt_content (used by extension dam_ttcontent )

tx_dam

Some of the data fields below are not yet used by the DAM. They will be marked as 'Not yet supported'. Their intended use in the future will be described none the less.

As with many record forms in the TYPO3 framework the backend form for the DAM record comes in a series of different field combinations dependent no the media_type of the file in question. The field description below covers every possible field even if no media_type will show all fields in one form.

uid

field

uid

description

Auto_increment field. Unique ID of the DAM record.

pid

field

pid

description

Database relation to the page record on which the record is located. uid=>pages

tstamp

field

tstamp

description

UNIX timestamp of the last changes to the record.

crdate

field

crdate

description

UNIX timestamp of the time of creation of the record.

cruser_id

field

cruser_id

description

Database relation to the be_user record who created the record. uid=>be_users

deleted

field

deleted

description

Boolean field with value 1 when the record has been deleted.

active – Not yet supported

field

active – Not yet supported

description

Boolean field. Only records with value 0 in this field will be available in the element browser.

sorting

field

sorting

description

Field holds sorting data for display in the DAM->List submodule.

hidden

field

hidden

description

Boolean field. Publication status of record. Use in frontend applications. See tslib_content::enableFields();

starttime

field

starttime

description

UNIX timestamp. Startdate for publication of record. Use in frontend applications. See tslib_content::enableFields();

endtime

field

endtime

description

UNIX timestamp. Enddate for publication of record. Use in frontend applications. See tslib_content::enableFields();

fe_group

field

fe_group

description

Database relation to a frontend user group. fe_group=>uid. Use in frontend applications. See tslib_content::enableFields();

media_type

field

media_type

description

One of the following main media types (list inspired by the dublin core). Text, Image, Audio, Video, Dataset, Interactive, Software, Model, Font, Collection, Service, Application, Undefined. Will be automatically assigned during indexing.

title

field

title

description

Text field. Title of document. Shown in Element browser and DAM->List submodule.

category

field

category

description

Pseudo field. Database relation to tx_dam_cat. Realized through tx_dam_mm_cat.

file_mime_type

field

file_mime_type

description

Corresponds to the field media_type. Not editable. One of the 12 possible mime_types.

file_mime_subtype

field

file_mime_subtype

description

Every mime_type can have one or more subtypes. Not editable. Example: mime_type Image, subtype jpeg. Mime_type and mime_subtype are displayed together in the field mime_type of the record editing form separated by a slash.

file_type

field

file_type

description

Holds the file type, normally the file ending. Not editable.

file_creator

field

file_creator

description

Holds information about the program used to create the file. i.e. Acrobat Distiller. Not editable.

file_type_version

field

file_type_version

description

Holds information about the version of the file type. i.e. “1.3” for PDF 1.3. Not editable.

file_name

field

file_name

description

Holds the name of the file in the filesystem of the server.

file_path

field

file_path

description

Holds the relative path to the file in the filesystem to the document root of the webserver. Not editable.

file_size

field

file_size

description

Filesize in bytes. Not editable.

file_mtime

field

file_mtime

description

UNIX timestamp.: The time the file was last modified. This function returns the time when the data blocks of a file were being written to, that is, the time when the content of the file was changed.

file_inode

field

file_inode

description

Integer: The inode number of the file

file_ctime

field

file_ctime

description

UNIX timestamp.: The time the file was last modified. In most Unix filesystems, a file is considered changed when its inode data is changed; that is, when the permissions, owner, group, or other metadata from the inode is updated.

file_hash - Not yet supported

field

file_hash - Not yet supported

description

Hash information about the related file.

file_status - Not yet supported

field

file_status - Not yet supported

description

file_orig_location - Not yet supported

field

file_orig_location - Not yet supported

description

file_orig_loc_desc - Not yet supported

field

file_orig_loc_desc - Not yet supported

description

thumb - Not yet supported

field

thumb - Not yet supported

description

Pseudofield. Used for display of a thumbnail in the backend.

thumb_path - Not yet supported

field

thumb_path - Not yet supported

description

Pseudofield. Used for display of a thumbnail in the backend.

file_dl_name

field

file_dl_name

description

Textfield to enter a name which should be linked to download a file in a potential frontend application.

file_usage - Not yet supported

field

file_usage - Not yet supported

description

Display of the content elements in which the record is used.

ident

field

ident

description

Text field. Could hold information like product number.

creator

field

creator

description

Text field. Could hold information about which person created the file. i.e. Photographer ...

keywords

field

keywords

description

Text field.

description

field

description

description

Text field.

abstract

field

abstract

description

Text field. Not editable. Hold excerpts from a text file.

search_content

field

search_content

description

Text field. Not editable. Will hold excerpts from a text file.

language

field

language

description

Select field. Language of the content of the file.

pages

field

pages

description

Integer Field. Number of pages in a text document.

publisher

field

publisher

description

Text field.

instructions

field

instructions

description

Text field.

date_cr

field

date_cr

description

Date field. Can hold information about the creation of a file independent of the creation of the actual document on the server. Example: the date on which a photo was taken as opposed to the date on which the actual jpg-document was scanned.

date_mod

field

date_mod

description

Date field. Can hold information about the changedate of a file independent of the changedate of the actual document on the server. Example: the date on which a digital artwork was modified as opposed to the date on which the actual document was saved last.

loc_desc

field

loc_desc

description

Text field. Example: description where a photo was taken.

loc_country

field

loc_country

description

Select field: see above - country

loc_city

field

loc_city

description

Select field: see above - city

hres

field

hres

description

Integer field. Horizontal resolution of an image.

vres

field

vres

description

Integer field. Vertical resolution of an image.

hpixels

field

hpixels

description

Integer field. Number of pixels an image is high.

vpixels

field

vpixels

description

Integer field. Number of pixels an image is wide.

color_space

field

color_space

description

Select field. One of the following values: RGB, CMYK, sRGB, CMY, YUV, indexed.

width

field

width

description

Float field.

height

field

height

description

Float field. Example: 29.7 (for the height of a DIN A format)

height_unit

field

height_unit

description

Select field. One of the following values: px, mm, cm, m, p. Defines the unit in which the values of width and height are given.

Multilanguage meta data

The DAM extension uses the language support given by the TYPO3 system. Please have a look in the TYPO3 documentation for more details.

The used concept is that translated records will be stored in the normal tx_dam table marked with a sys_language language uid and the record that holds the default language. This is configured in TCA:

$TCA['tx_dam'] = array(
        'ctrl' => array(
                'transOrigPointerField' => 'l18n_parent',
                'transOrigDiffSourceField' => 'l18n_diffsource',
                'languageField' => 'sys_language_uid',

The fields which shouldn't be translated are marked in TCA with

'file_name' => array(
        'l10n_mode' => 'exclude',
        'l10n_display' => 'defaultAsReadonly',

Those fields hold only valid data in the default language. That means when fetching a translated record the field file_name don't store any valid data and have to be ignored. Instead the field configured in

$TCA[$table]['ctrl']['transOrigPointerField']

have to be used to fetch the default data.

But how to know which fields are valid or not. This can be done with following function:

$lgOvlFields = tx_dam_db::getLanguageOverlayFields ('tx_dam', 'tx_dam_lgovl');

The field list can be used to create a self join with SQL which will look like this:

SELECT
        tx_dam.media_type, tx_dam.file_name,tx_dam.file_path,
        tx_dam_lgovl.title, tx_dam_lgovl.file_dl_name
FROM tx_dam,tx_dam as tx_dam_lgovl
WHERE tx_dam.pid IN (31)
        AND tx_dam.uid=tx_dam_lgovl.l18n_parent
        AND tx_dam_lgovl.sys_language_uid=1

A description how to use the API will come later.

Selections

Several components work together when a user compile a selection. The following summary gives an overview how it works on code level.

1. Clicking on an entry of a selections tree send a parameter like the following to the module:

...&SLCMD[SELECT][txdamMedia][jpg]=1
...&SLCMD[NOT][txdamMedia][png]=1

SLCMD is an abbreviation for Selection Command.

The first parameter key can be one of the following:

SELECT

a

SELECT

b

Send when clicking on the title. Clears the selection.

OR

a

OR

b

Send when clicking on the + icon.

AND

a

AND

b

Send when clicking on the = icon.

NOT

a

NOT

b

Send when clicking on the - icon.

DESELECT_ID

a

DESELECT_ID

b

Single records can be excluded from selections.

The second key defines a registered selection class.

The third key is defined by the selection tree itself and will be passed to the selection class to generate a SQL query part.

2. In the module base class tx_damSCbase the selection compiler class will be initialized. Following function detects the parameter array and merge it with the selection data from the user session.

tx_dam_selection::initSelection_getStored_mergeSubmitted()

3. The selection compiler is used to call for the parameters the right registered selection or selection tree class.

4. The selection class generates a small SQL query part depending on the passed parameters and the used selection type like SELECT or NOT.

5. In the module base class tx_damSCbase the compiled SQL query parts will be handed over to the query generator class tx_dam_querygen.

6. The query generator is ready to be used in modules to create the final SQL query. If wanted the query can be modified in beforehand with a count() statement for example.

DAM services

For more information on services see the services documentation. The indexing function of the DAM currently uses three service types: metaExtract , textExtract and textLang .

metaExtract

The metaExtract service is used to extract meta information form input files. For a functional implementation see the extensions cc_metaexif or cc_metaexec.

Subtypes of the metaExtract service

Every metaExtract service must be defined for one or more file types or all file types of one media type as subtypes. (See services documentation for the concept of subtypes.) Before the metaExtract service is used by the DAM, the DAM defines a file type and a media type for the file that is being indexed.

The first two lines of the code sample from the class dam/lib/class.tx_dam_indexing.php (from function getFileMetaInfo()) shows the way in which the metaExtract service is instantiated. The function tries to instantiate a service for the actual file type of the file (i.e. 'jpg'). If this is not possible it tries to instantiate a service for the general media type of the file (i.e. 'image').

Whether or not a specific service class is used for a given file depends on the subtypes defined for that service. A metaExtract service with the subtypes 'jpg, gif, png' will be used for any file fitting in one of these file types. A service with the subtypes 'image:*, mp3, doc' will be used for every file of the media type 'image' (if no more specific service for a given image type is available), for files of the type mp3 and for Word documents.

if (!is_object($serviceObj = t3lib_div::makeInstanceService('metaExtract',$fileType))) {
  $serviceObj = t3lib_div::makeInstanceService('metaExtract',$meta['fields']['media_type'].':*');
}
The metaExtract API

The API of the metaExtract service is very simple. The example below shows how the service is used by the DAM:

if (is_object($serviceObj)) {
  $serviceObj->setInputFile($absFile,$fileType);
       if ($serviceObj->process('','',array('meta'=>$meta))>0 AND (is_array($svmeta = $serviceObj->getOutput()))) {
        $meta = t3lib_div::array_merge_recursive_overrule($meta,$svmeta);
}
  $serviceObj->__destruct();
  unset($serviceObj);
}

In the second line of the code excerpt the indexing class defines the current file as the input file on which the service should operate. The function setInputFile() is part of the class t3lib/class.t3lib_svbase.php and defines the internal variables $this->inputFile and $this->inputType for the service class. The variable $this->inputFile holds an absolute file reference to the current file.

In line three the actual meta extraction is performed by the function process(). This function is the only function that needs to be implemented in the service itself.

The function needs to to two things:

A. It needs to fill the internal array $this->out['fields'] with data from the file for the respective fields of the database table tx_dam. (Example: giving a value $this->out['fields']['title'] will set the title field of the record, giving a value to $this->out['fields']['copyright'] will set the copyright field of the record.)

B. It needs to indicate whether or not its operations were successful. Meta information will only be taken if process() returns TRUE. (Have a look at the functions dedicated to error handling in class.t3lib_svbase.php)

The second operation within the if-clause of line three of the code excerpt consists in the actual retrieval of the data. The function getOutput() is part of the svbase class and returns the array $this->out of the service class.

textExtract

The textExtract service is used to extract text samples from text documents. In the context of the DAM the results of the service will be used to fill the fields 'search_content' and 'abstract' of the DAM record.For a functional implementation see the extensions cc_txtextexec or cc_txtextphp.

The textExtract service follows the same basic rules as the metaExtract service. Its relevance for specific file types is defined by subtypes. Defining a more general scope for a textExtract service by giving a media type as subtype ('text:*') is not possible in the DAM indexing context.

The input file of the service is once again defined by the function setInputFile().

The actual textinformation must be retrieved by the functino process() which again is the only function necessary in the specific service class. Other than in the metaExtract service the function process() need not return TRUE in order for the operation to go through.

As the extracted text is transferred by the function getOutput() again the function process() needs to save the textdata in the variable $this->out again. $this->out is no array in this context, but a text variable.

textLang

The textLang Service is used to detect the language of a text content. The detected language will be stored for files of media type text. For a functional implementation see the extensions cc_langguess or cc_textcat.

Debugging

The dam extension has a little support for debugging. Not much but ...

To activate it go into the Extension Managers detail view and enable following options.

img-7

((generated))

Option devel

Add an index rule that allows to delete the whole index before indexing.

Option debug

Enable debug output to backend modules like:

  • The indexing that runs inside an invisible iframe will become visible and show some information about the indexing process.
  • In the bottom of each module a debug output box can be displayed

img-8 When installed the BeInfo extension will be embedded.

img-9

The debug ouput can be extended with own content with:

$GLOBALS['SOBE']->debugContent['mykey'] = '<h4>My data</h4>'.t3lib_div::view_array($mydata);

img-10 Media: demo extension - 9