fabarea / metadata Goto Github PK

As apparently when the TCA from Overrides/* is cached the constant TYPO3_version is not available the workaround to display the "Access" label no longer works. As it seems to have worked with older versions maybe we should just reverse the condition.

Included ZendPdf library should be updated to newer version, which is PHP7 compatible

Hi,

I tried the extension and it worked :)
I noticed there are some metadata options which are not imported, like content_creation_date or content_modification_date.

PDF properties:

TYPO3 fields:

Would it be possible to import those values as well?

Thank you very much.

Although focal_length is a value calculated by a division

$metadata['focal_length'] = $parts[0] / $parts[1];

in

and thus can contain floats like 6.1 the database definition for the column is

focal_length int(4) unsigned DEFAULT '0' NOT NULL

in

At least with PostgreSQL this leads to a PDOException if such a value is encountered in FileStorageExtractionTask:

SQLSTATE[22P02]: Invalid text representation: 7 ERROR: invalid input syntax for integer: »6.1« (0) - {"exception":"PDOException: SQLSTATE[22P02]: Invalid text representation: 7 ERROR: invalid input syntax for integer: \u00bb6.1\u00ab in vendor\/doctrine\/dbal\/lib\/Doctrine\/DBAL\/Driver\/PDOStatement.php:91\nStack trace:
#0 vendor\/doctrine\/dbal\/lib\/Doctrine\/DBAL\/Driver\/PDOStatement.php(91): PDOStatement->execute(Array)
#1 vendor\/doctrine\/dbal\/lib\/Doctrine\/DBAL\/Connection.php(1008): Doctrine\\DBAL\\Driver\\PDOStatement->execute(Array)
#2 vendor\/doctrine\/dbal\/lib\/Doctrine\/DBAL\/Connection.php(678): Doctrine\\DBAL\\Connection->executeUpdate('UPDATE \"sys_fil...', Array, Array)
#3 typo3_src-8.7.10\/typo3\/sysext\/core\/Classes\/Database\/Connection.php(357): Doctrine\\DBAL\\Connection->update('\"sys_file_metad...', Array, Array, Array)
#4 typo3_src-8.7.10\/typo3\/sysext\/core\/Classes\/Resource\/Index\/MetaDataRepository.php(193): TYPO3\\CMS\\Core\\Database\\Connection->update('sys_file_metada...', Array, Array, Array)
#5 typo3_src-8.7.10\/typo3\/sysext\/core\/Classes\/Resource\/Index\/Indexer.php(154): TYPO3\\CMS\\Core\\Resource\\Index\\MetaDataRepository->update(1379, Array)
#6 typo3_src-8.7.10\/typo3\/sysext\/core\/Classes\/Resource\/Index\/Indexer.php(114): TYPO3\\CMS\\Core\\Resource\\Index\\Indexer->extractMetaData(Object(TYPO3\\CMS\\Core\\Resource\\File))
#7 typo3_src-8.7.10\/typo3\/sysext\/scheduler\/Classes\/Task\/FileStorageExtractionTask.php(49): TYPO3\\CMS\\Core\\Resource\\Index\\Indexer->runMetaDataExtraction(100)
#8 typo3_src-8.7.10\/typo3\/sysext\/scheduler\/Classes\/Scheduler.php(181): TYPO3\\CMS\\Scheduler\\Task\\FileStorageExtractionTask->execute()

component="TYPO3.CMS.Scheduler.Task.FileStorageExtractionTask": A Task Exception was captured: An exception occurred while executing
UPDATE "sys_file_metadata" 
SET "pid" = ?, "tstamp" = ?, "crdate" = ?, "cruser_id" = ?, "sys_language_uid" = ?, "l10n_parent" = ?, "l10n_diffsource" = ?, "t3ver_oid" = ?, "t3ver_id" = ?, "t3ver_wsid" = ?, "t3ver_label" = ?, "t3ver_state" = ?, "t3ver_stage" = ?, "t3ver_count" = ?, "t3ver_tstamp" = ?, "t3ver_move_id" = ?, "t3_origuid" = ?, "file" = ?, "title" = ?, "width" = ?, "height" = ?, "description" = ?, "alternative" = ?, "visible" = ?, "status" = ?, "keywords" = ?, "caption" = ?, "creator_tool" = ?, "download_name" = ?, "creator" = ?, "publisher" = ?, "source" = ?, "copyright" = ?, "location_country" = ?, "location_region" = ?, "location_city" = ?, "latitude" = ?, "longitude" = ?, "ranking" = ?, "content_creation_date" = ?, "content_modification_date" = ?, "note" = ?, "unit" = ?, "duration" = ?, "color_space" = ?, "pages" = ?, "language" = ?, "fe_groups" = ?, "l10n_state" = ?, "categories" = ?, "copyright_notice" = ?, "aperture_value" = ?, "shutter_speed_value" = ?, "iso_speed_ratings" = ?, "camera_model" = ?, "focal_length" = ?, "flash" = ?, "metering_mode" = ?, "horizontal_resolution" = ?, "vertical_resolution" = ?
WHERE "uid" = ?' with params ["0", 1519306909, "1508339162", "3", "0", "0", "", "0", "0", "0", "", "0", "0", "0", "0", "0", "0", "1379", null, "900", "675", null, null, "1", "", null, null, "Adobe Photoshop CS3 Macintosh", "", "", "", "", null, "", "", "", "0.00000000000000", "0.00000000000000", "0", "1264852285", "1264957253", null, "px", "0", "sRGB", "0", "", null, null, "0", "", "2.9", "1\/999", "80", "Canon PowerShot G11", "6.1", "16", "5", "72", "72", 1379]

Metadata is first fetched from EXIF and the overridden by IPTC:
https://github.com/fabarea/metadata/blob/master/Classes/Index/ImageMetadataExtractor.php#L140

content_creation_date is fetched in EXIF using:
https://github.com/fabarea/metadata/blob/master/Classes/Index/ImageMetadataExtractor.php#L229

and in IPTC using
https://github.com/fabarea/metadata/blob/master/Classes/Index/ImageMetadataExtractor.php#L66
and
https://github.com/fabarea/metadata/blob/master/Classes/Index/ImageMetadataExtractor.php#L348

Consider a picture from an iOS device which contains EXIF and IPTC.

• EXIF contains a parseable DateTimeOriginal and DateTimeDigitized
string(19) "2020:12:02 14:49:02"
  which can be parsed to a UNIX timestamp int(1606916942) and be properly used in TYPO3.
• IPTC only defines a "Date created" instead of a timestamp, e.g. 20201202 see https://iptc.org/std/photometadata/specification/IPTC-PhotoMetadata#date-created - thus contains less information (missing the time).

Currently there are two issues:

1. An IPTC "Date created" is not parsed with strtotime() and leads to wrong timestamps written to the database (the database columns contains the unparsed 20201202 instead of 1606863600.
2. The correctly parsed and more accurate EXIF timestamp int(1606916942) is overridden.

The pdf indexer fails if the file contains an array of keywords in the pdf metadata block.
In this cails the scheduler task fails and an entry to the typo3 log is written. Every index run.
The error occurs when the values of the metadata block are parsed and converted to xml.
This should be improved to handle single values and also arrays.
The second image shows the array $pdf->properties.

TYPO3: 10.4.17
PHP: 7.4.20
EXT:metadata: 4.0.0

In TYPO3 7.4.0 the tasks fails with an error message:

Serialization of 'Closure' is not allowed

TCA allows 255 characters, DB field is VARCHAR(32) and therefore removes content.

I'm not sure why this was set to 32.
https://www.iptc.org/std/photometadata/specification/IPTC-PhotoMetadata#credit-line

Avoid scheduler task of FAL metadata extract to fail to execute successfully, regarding type and value of incoming EXIF data. In some cases, existing EXIF data could contain zero values (such as resolution/fraction, shutter speed or GPS coordinates).
3 commits submited as pull-requests.

by overwriting $tca['types'] for the filetype image the configuration for showing the categories ("--div--;LLL:EXT:lang/locallang_tca.xlf:sys_category.tabs.category, categories,") gets removed.
This results in being unable to edit categories for images in the backend.

for using with TYPO3 8 LTS - what is to do ???

Hi,

I created a document with Adobe InDesign Cloud 2019 and added all kinds of metadata. I exported the document as PDF and when I import the PDF into TYPO3 the keywords are missing in the TYPO3 backend. Nothing gets imported.

When I edit the PDF with Adobe Acrobat the keywords are present so I wonder why it's not working with the exported InDesign file.

I use TYPO3 9.5.13.

The EXIF definition for ColorSpace defined 65535 as "Uncalibrated".
https://sno.phy.queensu.ca/~phil/exiftool/TagNames/EXIF.html

As this is currently not defined in the $colorSpaceToNameMapping in

the method getColorSpace() in

returns the bare value 65535 and sets it as color_space.

But the SQL definition for this column in filemetadata is color_space varchar(4) DEFAULT '', which only allows 4 characters. So when trying to add metadata for a file with a color_space value of 65535 at least on PostgreSQL an SQL PDOException is thrown as the value is too long for the field.

content_creation_date is defined as int(11): content_creation_date int(11) unsigned DEFAULT '0'

strtotime('0000:00:00 00:00:00') returns an integer -62169987208 which is out of range (at least for PostgreSQL):

Numeric value out of range: 7 ERROR:  Value »-62169987208« is out of range for type integer

Are there plans to support TYPO3 v10?

When translating the metadata, default values of the TCA is used. As the TCA is configured to have -1, it causes problems.

On PostgreSQL an SQL exception is thrown if an image does not contain a GPS coordinate because lat/lon are defined as

latitude decimal(24,14) DEFAULT '0.00000000000000',
longitude decimal(24,14) DEFAULT '0.00000000000000',

in TYPO3 Core, but the current implementation in EXT:metadata sets an empty string '' as lat/lon if no coordinate is found in https://github.com/fabarea/metadata/blob/master/Classes/Index/ImageMetadataExtractor.php#L418

As an empty string is not numeric this fails on PostgreSQL (MySQL tolerates it).

Instead of
'2#115' => 'publisher',

should it not be
'2#115' => 'source',

Otherwise IPTC source metadata is saved in publisher instead of source.
Or is this on purpose? I'm not that deep into metadata stuff....

Hello,

looking on TER I found this extension where it is defined that supported up to TYPO3 8.
On GitHub, however, it would seem to also support TYPO3 9.
Are you developing and it hasn't been released yet?
Do you already have the expected time for the possible release?

Thanks for your attention.

regards
Abramo

If EXT:metadata is installed I can't see my custom added columns in file mata data TCA.

I add my columns like this in sys_file_metadata.php:

\TYPO3\CMS\Core\Utility\ExtensionManagementUtility::addTCAcolumns(
    'sys_file_metadata',
    [
        'myField' => [
            'exclude' => 0,
            'label'   =>'myFieldLabel',
            'config'  => [
                'type' => 'input',
                'size' => 30,
                'max'  => 1000,
                'eval' => 'trim',
            ],
        ],
    ]
);

\TYPO3\CMS\Core\Utility\ExtensionManagementUtility::addToAllTCAtypes(
    'sys_file_metadata',
    ', --div--;myDivLabel, myField,',
    '',
    'after'
);

The reason is, that EXT:metadata uses ArrayUtility::mergeRecursiveWithOverrule in metadata/Configuration/TCA/Overrides/sys_file_metadata.php

\TYPO3\CMS\Core\Utility\ArrayUtility::mergeRecursiveWithOverrule($GLOBALS['TCA']['sys_file_metadata'], $tca);

I got some troubles with inserting GPS coordinates when lattitude/longitude aren't present in metadata...

Following change in function parseGpsCoordinate worked for me:
return ($value === null || $value === '' || $value == 0) ? '0.00000000000000' : (string)$value;

instead of:
return $value === null ? null : (string)$value;

Best regards,
Roman

In ImageMetadataExtractor.php on line 115
$file->getForLocalProcessing();
is called.
Which creates persistent fal-tmp Files in typ3temp, see: https://forge.typo3.org/issues/62098
As write access is not needed here,
$file->getForLocalProcessing(FALSE);
works too and fixes this problem.

I think this is obsolete?

Version 2.4.0-dev is not compatible with current TYPO3 CMS 9.5, which should be to support upgrade.

If you install current version in TYPO3 CMS version 9.5.5, then

• extension configuration check uses deprecated TYPO3_CONF_VARS key
• extension configuration breaks by class loading from extension configuration user-function which renders result for PHP extension check (exif & iptc)
• extension configuration will not render with new introduced css classes for success/warning

I will add a pull request, which solves these issue..