nextcloud / previewgenerator Goto Github PK
View Code? Open in Web Editor NEWNextcloud app to do preview generation in the background.
Home Page: https://apps.nextcloud.com/apps/previewgenerator
License: GNU Affero General Public License v3.0
Nextcloud app to do preview generation in the background.
Home Page: https://apps.nextcloud.com/apps/previewgenerator
License: GNU Affero General Public License v3.0
According to the desciption
Listen to events that a file has been writen or modified and store it in the database
an tests with my NextCloud installation it seems, that preview:generate-all
does not include external storage. Is that right?
If so: It would be a nice feature for future versions, to generate previews for external storage content (e.g. with an additional trigger like ./occ preview:generate-all [include-external]
).
I've added my photo collection to my NextCloud (runnig on a Raspberry PI 3) via SMB...and the on-demand generation speed of the thumbnail pics is not really satisfying...
Thanks a lot in advance!
Hi, I love the concept of this app but I'm bit confused how this app works.
I have created a test user (username: photos.db), uploaded 800MB folder full of photos.
then initiate the scan:
sudo -u www php /www/nextcloud/occ preview:generate-all photos.db
When that finished the appdata/preview folder was 500MB in size - that's nearly as big as the original folder. Furthermore when I login to nextcloud and switch to galery view I cannot see any speed improvment in the preview generation.
Am I missing something here?
sudo -u www-data php occ preview:pre-generate
↓
Getötet
That's all what the app displays. No log in nextcloud.log or Apache as well. Just "killed".
It comes from a big jpg file which is created by Hugin panorama editor.
The preview generator app should parse the image size before processing it and ignore the preview generation if the image is too big.
How are the limitations at the moment and how can i adjust them?
$ sudo -u oc_user php5 /www/owncloud/occ preview:generate
**** Unrecoverable error in xref!
Error: /rangecheck in resolveR
Operand stack:
--dict:3/3(L)-- --dict:4/4(L)-- --dict:4/4(L)-- --dict:4/4(L)-- 4252440 1050 z▒▒▒
Execution stack:
%interp_exit .runexec2 --nostringval-- --nostringval-- --nostringval-- 2 %stopped_push --nostringval-- --nostringval-- --nostringval--
false 1 %stopped_push 1910 1 3 %oparray_pop 1909 1 3 %oparray_pop 1893 1 3 %oparray_pop --nostringval-- --nostringval-- --nostringval-- --nostringval-- --nostringval-- --nostringval-- --nostringval-- --nostringval-- %array_continue --nostringval-- --nostringval-- --nostringval-- %array_continue --nostringval-- --nostringval-- --nostringval-- %array_continue --nostringval-- --nostringval-- --nostring
val-- %array_continue --nostringval-- --nostringval-- --nostringval--
Dictionary stack:
--dict:1174/1684(ro)(G)-- --dict:1/20(G)-- --dict:82/200(L)-- --dict:82/200(L)-- --dict:109/127(ro)(G)-- --dict:293/300(ro)(G)-- --dict:21/31(
L)-- --dict:5/10(L)--
Current allocation mode is local
GPL Ghostscript 9.06: Unrecoverable error, exit code 1
Error: /typecheck in defineresource
Operand stack:
--dict:10/19(L)-- false DefaultGray DeviceGray ColorSpace DefaultGray DeviceGray
Execution stack:
%interp_exit .runexec2 --nostringval-- --nostringval-- --nostringval-- 2 %stopped_push --nostringval-- --nostringval-- --nostringval--
false 1 %stopped_push 1910 1 3 %oparray_pop 1909 1 3 %oparray_pop 1893 1 3 %oparray_pop --nostringval-- --nostringval-- 2
1 1 --nostringval-- %for_pos_int_continue --nostringval-- --nostringval-- --nostringval-- 1862 5 7 %oparray_pop defineresource %errorexec_pop --nostringval--
Dictionary stack:
--dict:1175/1684(ro)(G)-- --dict:1/20(G)-- --dict:82/200(L)-- --dict:82/200(L)-- --dict:109/127(ro)(G)-- --dict:293/300(ro)(G)-- --dict:25/31(
L)-- --dict:16/20(ro)(G)--
Current allocation mode is local
Last OS error: No such file or directory
GPL Ghostscript 9.06: Unrecoverable error, exit code 1
Error: /undefined in ▒d
Operand stack:
Execution stack:
%interp_exit .runexec2 --nostringval-- --nostringval-- --nostringval-- 2 %stopped_push --nostringval-- --nostringval-- --nostringval-- false 1 %stopped_push 1910 1 3 %oparray_pop 1909 1 3 %oparray_pop 1893 1 3 %oparray_pop 1787 1 3 %oparray_pop --nostringval-- %errorexec_pop .runexec2 --nostringval-- --nostringval-- --nostringval-- 2 %stopped_push --nostringval--
Dictionary stack:
--dict:1174/1684(ro)(G)-- --dict:0/20(G)-- --dict:77/200(L)--
Current allocation mode is local
Current file position is 5
GPL Ghostscript 9.06: Unrecoverable error, exit code 1
PHP Fatal error: imagecreatefromjpeg(): gd-jpeg: JPEG library reports unrecoverable error: Unsupported marker type 0xb9 in /www/owncloud/lib/private/legacy/image.php on line 538
Hello,
I manually ran "generate all", then I manually ran pre-generate, and I got the following exception:
php -f /var/www/nextcloud/occ preview:pre-generate
An unhandled exception has been thrown:
Error: Call to a member function file_get_contents() on null in /var/www/nextcloud/lib/private/Files/Filesystem.php:719
Stack trace:
#0 /var/www/nextcloud/lib/private/legacy/image.php(597): OC\Files\Filesystem::file_get_contents('/var/www/nextcl...')
#1 /var/www/nextcloud/lib/private/Preview/MP3.php(80): OC_Image->loadFromFile('/var/www/nextcl...')
#2 /var/www/nextcloud/lib/private/Preview/MP3.php(64): OC\Preview\MP3->getNoCoverThumbnail()
#3 /var/www/nextcloud/lib/private/Preview/GeneratorHelper.php(54): OC\Preview\MP3->getThumbnail('/files/Document...', 2048, 2048, false, Object(OC\Files\View))
I'm running Nextcloud 11.0.2 (stable), with defaults apps and previewgenerator activated. previewgenerator 1.0.3.
After debugging a while, I found the file that triggers the exception, it's a mp3 file from a chinese book I bought, here it is:
Thanks for the great pluging !
I am on shared hosting. Is there a way to trigger this using webcron? How does this app actually work? Will it generate previews for new images only? What are the resolutions of the generated previews? Only the default 2048x2048?
Command runs fine and builds a large cache of previews in the appdata_xxxx/preview folder, but none of these files appear to be used by nextcloud, and the gallery app makes on-demand previews in the user/thumbnails folder instead of using the previews generated by this app. Have I configured something incorrectly?
I recently renamed my data directory from /data/owncloud to /data/nextcloud. This had the unfortunate consequence of making all the previously generated previews invalid and forcing me to re-generate them all (which is quite unpleasant since it takes days...). I guess that the paths are coded as full paths, therefore making such as data directory rename quite unpleasant. If paths were stored relative to the data directory root, then the previews would still be valid and lots of time would be saved...
I tried reading the README and glancing at the code, neither gave me a clear picture on the difference between both. Both seem to generate previews?
This repo could use a README :)
Hello!
The "generate-all" command seems to ignore the global log level of Nextcloud.
root@red:htdocs/config# grep level config.php -H
config.php: 'loglevel' => 2,
Used previewgenerator-command:
What's happening with the preview of deleted files?
Do they stay in the preview folder or are they deleted by preview:pre-generate?
If they are not deleted, should the content of the previev-folder be deleted time by time and new created by running preview:generate-all?
In my NC setup I have an external storage /path/to/my/photos
setup for all users. When previewgenerator
generates image caches, will it generate one for each user or will it only generate one cache and share it for all the users?
occ preview:pre-generate
command generates an exception:
An unhandled exception has been thrown:
Error: Call to a member function file_get_contents() on null in /var/www/kolosowscy.cloud/htdocs/lib/private/Files/Filesystem.php:719
Stack trace:
#0 /var/www/kolosowscy.cloud/htdocs/lib/private/legacy/image.php(597): OC\Files\Filesystem::file_get_contents('/var/www/koloso...')
#1 /var/www/kolosowscy.cloud/htdocs/lib/private/Preview/MP3.php(80): OC_Image->loadFromFile('/var/www/koloso...')
#2 /var/www/kolosowscy.cloud/htdocs/lib/private/Preview/MP3.php(64): OC\Preview\MP3->getNoCoverThumbnail()
#3 /var/www/kolosowscy.cloud/htdocs/lib/private/Preview/GeneratorHelper.php(54): OC\Preview\MP3->getThumbnail('/files/Dokument...', 2048, 2048, false, Object(OC\Files\View))
#4 /var/www/kolosowscy.cloud/htdocs/lib/private/Preview/Generator.php(155): OC\Preview\GeneratorHelper->getThumbnail(Object(OC\Preview\MP3), Object(OC\Files\Node\File), 2048, 2048)
#5 /var/www/kolosowscy.cloud/htdocs/lib/private/Preview/Generator.php(108): OC\Preview\Generator->getMaxPreview(Object(OC\Files\SimpleFS\SimpleFolder), Object(OC\Files\Node\File), 'audio/mpeg')
#6 /var/www/kolosowscy.cloud/htdocs/lib/private/PreviewManager.php(182): OC\Preview\Generator->getPreview(Object(OC\Files\Node\File), 32, 32, true, 'fill', 'audio/mpeg')
#7 /var/www/kolosowscy.cloud/htdocs/apps/previewgenerator/lib/Command/PreGenerate.php(158): OC\PreviewManager->getPreview(Object(OC\Files\Node\File), 32, 32, true)
#8 /var/www/kolosowscy.cloud/htdocs/apps/previewgenerator/lib/Command/PreGenerate.php(181): OCA\PreviewGenerator\Command\PreGenerate->processFile(Object(OC\Files\Node\File))
#9 /var/www/kolosowscy.cloud/htdocs/apps/previewgenerator/lib/Command/PreGenerate.php(183): OCA\PreviewGenerator\Command\PreGenerate->processFolder(Object(OC\Files\Node\Folder))
#10 /var/www/kolosowscy.cloud/htdocs/apps/previewgenerator/lib/Command/PreGenerate.php(150): OCA\PreviewGenerator\Command\PreGenerate->processFolder(Object(OC\Files\Node\Folder))
#11 /var/www/kolosowscy.cloud/htdocs/apps/previewgenerator/lib/Command/PreGenerate.php(113): OCA\PreviewGenerator\Command\PreGenerate->processRow(Array)
#12 /var/www/kolosowscy.cloud/htdocs/apps/previewgenerator/lib/Command/PreGenerate.php(91): OCA\PreviewGenerator\Command\PreGenerate->startProcessing()
#13 /var/www/kolosowscy.cloud/htdocs/3rdparty/symfony/console/Command/Command.php(256): OCA\PreviewGenerator\Command\PreGenerate->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#14 /var/www/kolosowscy.cloud/htdocs/3rdparty/symfony/console/Application.php(818): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#15 /var/www/kolosowscy.cloud/htdocs/3rdparty/symfony/console/Application.php(186): Symfony\Component\Console\Application->doRunCommand(Object(OCA\PreviewGenerator\Command\PreGenerate), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#16 /var/www/kolosowscy.cloud/htdocs/3rdparty/symfony/console/Application.php(117): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#17 /var/www/kolosowscy.cloud/htdocs/lib/private/Console/Application.php(169): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#18 /var/www/kolosowscy.cloud/htdocs/console.php(90): OC\Console\Application->run()
#19 /var/www/kolosowscy.cloud/htdocs/occ(11): require_once('/var/www/koloso...')
#20 {main}
after few hours i get
root@omv:/var/www/nextcloud# sudo -uowncloud php ./occ preview:generate-all PHP Fatal error: imagecreatefromjpeg(): gd-jpeg: JPEG library reports unrecoverable error: Unsupported marker type 0x6f in /var/www/nextcloud/lib/private/legacy/image.php on line 538
nexcloud v11.0.0
previewgenerator v1.0.1
php v5.6.29
I'm not 100% sure, but I think that error started appearing after the last update of the plugin. The version I'm using is: 1.0.5
I've got a cronjub which runs:
/var/www/html/cloud/occ preview:pre-generate
it then generates following error message.
PHP Fatal error: Call to a member function file_get_contents() on null in /var/www/html/cloud/lib/private/Files/Filesystem.php on line 719
The last few months it was running well. But as I'm said, it happened shortly after the last update, but I can't say right now if directly, or if maybe some files were uploaded which generate that error.
When i execute the command: "occ preview:pre-generate" i am getting the follwoing error:
imagecreatefromjpeg(): gd-jpeg: JPEG library reports unrecoverable error: Invalid JPEG file structure: two SOI markers in /lib/private/legacy/image.php on line 538
Not sure what it means but i comes everytime when i run the preview generator and it seems that because of it no more previews are beeing generated.
Hi there, have been trying to run preview:generate-all or preview:pre-generate and I receive this error:
root@snappy-nextcloud:/var/snap/nextcloud/current/nextcloud/extra-apps# nextcloud.occ preview:pre-generate
The process control (PCNTL) extensions are required in case you want to interrupt long running commands - see http://php.net/manual/en/book.pcntl.php
[OC\Encryption\Exceptions\DecryptionFailedException]
Can not decrypt this file, probably this is a shared file. Please ask the file owner to reshare the file with y
ou.
preview:pre-generate
Let me know if there's any other info I can send you, and thanks for your help.
The preview pictures are located in ~/nextcloud_data/appdata_XXXXXXX/preview
Each file has his own subfolder for the preview pictures in there.
The count of subfolders in a folder is limited by the inodes of the system. So if you have really lots of files, the chance to run out of inodes is growing. The result would be that you can't ls -al
anymore in the preview folder.
I would suggest that the subfolders for the preview pictures are not stored in one huge folder, rather divided into a certain count of subfolders which contains subfolders as well.
Similar like the cache system of Squid proxy.
squid -z
ls -al /var/spool/squid
I recognized that the generate-all command does not verify if its a shared folder. So it scanned all shared folders for every user again. Maybe when there is some time in the future, this could be a little speed up.
I'm getting this error when i give the 'nextcloud.occ preview:pre-generate' command:
ERROR: ld.so: object '/usr/lib/arm-linux-gnueabihf/libarmmem.so' from /etc/ld.so.preload cannot be preloaded (cannot open shared object file): ignored.
Hey, thanks for this app!
When running any command I get this error:
root@cloud:~# sudo -u www-data php /var/www/nextcloud/occ preview:pre-generate
[OC\Encryption\Exceptions\DecryptionFailedException]
Can not decrypt this file, probably this is a shared file. Please ask the file owner to reshare the file with you.
preview:pre-generate
PHP Fatal error: Uncaught Error: Access to undeclared static property: OC\Files\Filesystem::$normalizedPathCache in /var/www/nextcloud/lib/private/Files/Filesystem.php:806
Stack trace:
#0 /var/www/nextcloud/lib/private/Files/View.php(2018): OC\Files\Filesystem::normalizePath('/usr/files/Ph...')
#1 /var/www/nextcloud/lib/private/Files/View.php(1156): OC\Files\View->unlockFile('/files/Photos/M...', 1)
#2 [internal function]: OC\Files\View->OC\Files\{closure}()
#3 /var/www/nextcloud/apps/files_external/3rdparty/icewind/streams/src/CallbackWrapper.php(109): call_user_func(Object(Closure))
#4 [internal function]: Icewind\Streams\CallbackWrapper->stream_close()
#5 {main}
thrown in /var/www/nextcloud/lib/private/Files/Filesystem.php on line 806
Anything to do with decryption?
Instead of displaying the filename it is displaying the producer → no way to debug :-(
Here the PDF file in question
https://mega.nz/#!9A0E2BYA!9OG26578lm03HqaBHvqE8xtUPHWmwlWTAHeSfkizLfA
If i do the command ./occ preview:generate-all, i get this error:
[OC\Encryption\Exceptions\DecryptionFailedException]
Can not decrypt this file, probably this is a shared file. Please ask the file owner to reshare the file with you.
I have deleted all shared links and encyption is all disabled. What can I do?
Nextcloud: "12.0.0.16"
{
"id": "ocysxstxaaf8",
"items": [
[
"server",
"version",
"12.0.0.16"
],
[
"server",
"code",
"other"
],
[
"server",
"enable_avatars",
"yes"
],
[
"server",
"enable_previews",
"yes"
],
[
"server",
"memcache.local",
"\\OC\\Memcache\\APCu"
],
[
"server",
"memcache.distributed",
"none"
],
[
"server",
"asset-pipeline.enabled",
"yes"
],
[
"server",
"filelocking.enabled",
"yes"
],
[
"server",
"memcache.locking",
"none"
],
[
"server",
"debug",
"no"
],
[
"server",
"cron",
"cron"
],
[
"php",
"version",
"7.1.4"
],
[
"php",
"memory_limit",
1073741824
],
[
"php",
"max_execution_time",
3600
],
[
"php",
"upload_max_filesize",
17179869184
],
[
"database",
"type",
"mysql"
],
[
"database",
"version",
"5.7.18"
],
[
"database",
"size",
417939456
],
[
"apps",
"EagleEye",
"disabled"
],
[
"apps",
"activity",
"2.5.1"
],
[
"apps",
"activitylog",
"disabled"
],
[
"apps",
"audioplayer",
"disabled"
],
[
"apps",
"bookmarks",
"disabled"
],
[
"apps",
"calendar",
"disabled"
],
[
"apps",
"checksum",
"disabled"
],
[
"apps",
"comments",
"1.2.0"
],
[
"apps",
"contacts",
"disabled"
],
[
"apps",
"dav",
"1.3.0"
],
[
"apps",
"delete_confirmation",
"disabled"
],
[
"apps",
"documents",
"disabled"
],
[
"apps",
"encryption",
"1.6.0"
],
[
"apps",
"external",
"2.0.0"
],
[
"apps",
"federatedfilesharing",
"1.2.0"
],
[
"apps",
"federation",
"1.2.0"
],
[
"apps",
"files",
"1.7.2"
],
[
"apps",
"files_antivirus",
"disabled"
],
[
"apps",
"files_downloadactivity",
"disabled"
],
[
"apps",
"files_embeddedvideo",
"disabled"
],
[
"apps",
"files_epubviewer",
"disabled"
],
[
"apps",
"files_external",
"1.3.0"
],
[
"apps",
"files_locking",
"disabled"
],
[
"apps",
"files_mv",
"disabled"
],
[
"apps",
"files_pdfviewer",
"1.1.1"
],
[
"apps",
"files_reader",
"1.0.4"
],
[
"apps",
"files_sharing",
"1.4.0"
],
[
"apps",
"files_texteditor",
"2.4.1"
],
[
"apps",
"files_trashbin",
"1.2.0"
],
[
"apps",
"files_versions",
"1.5.0"
],
[
"apps",
"files_videojs-sublime",
"disabled"
],
[
"apps",
"files_videoplayer",
"1.1.0"
],
[
"apps",
"files_videoviewer",
"disabled"
],
[
"apps",
"files_videoviewerplus",
"disabled"
],
[
"apps",
"firstrunwizard",
"2.1"
],
[
"apps",
"flowupload",
"disabled"
],
[
"apps",
"folderplayer",
"disabled"
],
[
"apps",
"gallery",
"17.0.0"
],
[
"apps",
"galleryplus",
"disabled"
],
[
"apps",
"html5_videoplayer",
"disabled"
],
[
"apps",
"libreonline",
"disabled"
],
[
"apps",
"logreader",
"2.0.0"
],
[
"apps",
"lookup_server_connector",
"1.0.0"
],
[
"apps",
"mail",
"disabled"
],
[
"apps",
"music",
"disabled"
],
[
"apps",
"nextant",
"disabled"
],
[
"apps",
"nextcloud_announcements",
"1.1"
],
[
"apps",
"notifications",
"2.0.0"
],
[
"apps",
"ocdownloader",
"disabled"
],
[
"apps",
"ownbackup",
"disabled"
],
[
"apps",
"password_policy",
"1.2.2"
],
[
"apps",
"pinit",
"disabled"
],
[
"apps",
"previewgenerator",
"1.0.6"
],
[
"apps",
"provisioning_api",
"1.2.0"
],
[
"apps",
"rainloop",
"disabled"
],
[
"apps",
"richdocuments",
"disabled"
],
[
"apps",
"search_lucene",
"disabled"
],
[
"apps",
"serverinfo",
"1.2.0"
],
[
"apps",
"sharebymail",
"1.2.0"
],
[
"apps",
"storagecharts2",
"disabled"
],
[
"apps",
"survey_client",
"1.0.0"
],
[
"apps",
"systemtags",
"1.2.0"
],
[
"apps",
"templateeditor",
"0.2"
],
[
"apps",
"theming",
"1.3.0"
],
[
"apps",
"twofactor_backupcodes",
"1.1.0"
],
[
"apps",
"updatenotification",
"1.2.0"
],
[
"apps",
"updater",
"disabled"
],
[
"apps",
"user_external",
"0.4"
],
[
"apps",
"user_webdavauth",
"disabled"
],
[
"apps",
"workflowengine",
"1.2.0"
],
[
"stats",
"num_files",
282365
],
[
"stats",
"num_users",
3
],
[
"stats",
"num_storages",
55
],
[
"stats",
"num_storages_local",
17
],
[
"stats",
"num_storages_home",
3
],
[
"stats",
"num_storages_other",
35
],
[
"stats",
"num_comments",
0
],
[
"stats",
"num_comment_markers",
0
],
[
"stats",
"num_systemtags",
0
],
[
"stats",
"num_systemtags_mappings",
0
],
[
"files_sharing",
"num_shares",
13
],
[
"files_sharing",
"num_shares_user",
0
],
[
"files_sharing",
"num_shares_groups",
0
],
[
"files_sharing",
"num_shares_link",
13
],
[
"files_sharing",
"num_shares_link_no_password",
13
],
[
"files_sharing",
"num_fed_shares_sent",
0
],
[
"files_sharing",
"num_fed_shares_received",
0
],
[
"files_sharing",
"permissions_3_1",
"10"
],
[
"files_sharing",
"permissions_3_15",
"3"
],
[
"encryption",
"enabled",
"no"
],
[
"encryption",
"default_module",
"yes"
]
]
}
Hi,
after 2 days that the process runs for hours, I have start the command with verbose flag:
occ preview:pre-generate --force -vvv
it don't stop and seems analyze continuously and circularly all the images.
NC 11.0.1 on Ubuntu 16.04.1
The occ
command preview:generate-all
truncates the filenames in nextcloud.log, therefore debugging is impossible at the moment.
sudo -u www-data php occ preview:generate-all
tail -f nextcloud.log
MP3->getThumbnail('\/files\/Archiv\/K...', 2048, 2048, false, O
https://help.nextcloud.com/t/preview-generate-all-not-working/8190/4
Hi All,
just after few seconds after I run
sudo -u www-data php /var/www/nextcloud/occ preview:pre-generate
I had an error:
PHP Fatal error: imagecreatefromjpeg(): gd-jpeg: JPEG library reports unrecoverable error: Unsupported marker type 0xba in /var/www/nextcloud/lib/private/legacy/image.php on line 538
My system is:
PHP 7.0.13
mysql 5.7.17
Nexcloud 11.0.1
Hi,
The idea of the app is very good.
why can't previewgenerator work if encryption is enabled?
To make Nextcloud secure, https is the good option...
Thanks for the answer
I downloaded and installed your app from the appstore inside NC 11RC1. Everything worked fine even the preview:pre-generate command worked perfectly. I tried the generate-all feature and that was the error message I got.
occ preview:generate-all
An unhandled exception has been thrown:
Error: Call to a member function getPath() on null in /var/www/magicbroccoli.de/nextcloud/lib/private/Preview/GeneratorHelper.php:72
Stack trace:
#0 /var/www/magicbroccoli.de/nextcloud/lib/private/Preview/GeneratorHelper.php(53): OC\Preview\GeneratorHelper->getViewAndPath(NULL)
#1 /var/www/magicbroccoli.de/nextcloud/lib/private/Preview/Generator.php(155): OC\Preview\GeneratorHelper->getThumbnail(Object(OC\Preview\MP3), Object(OC\Files\Node\File), 1024, 1024)
#2 /var/www/magicbroccoli.de/nextcloud/lib/private/Preview/Generator.php(108): OC\Preview\Generator->getMaxPreview(Object(OC\Files\SimpleFS\SimpleFolder), Object(OC\Files\Node\File), 'audio/mpeg')
#3 /var/www/magicbroccoli.de/nextcloud/lib/private/PreviewManager.php(182): OC\Preview\Generator->getPreview(Object(OC\Files\Node\File), 32, 32, true, 'fill', 'audio/mpeg')
#4 /var/www/magicbroccoli.de/nextcloud/apps/previewgenerator/lib/Command/Generate.php(167): OC\PreviewManager->getPreview(Object(OC\Files\Node\File), 32, 32, true)
#5 /var/www/magicbroccoli.de/nextcloud/apps/previewgenerator/lib/Command/Generate.php(155): OCA\PreviewGenerator\Command\Generate->parseFile(Object(OC\Files\Node\File))
#6 /var/www/magicbroccoli.de/nextcloud/apps/previewgenerator/lib/Command/Generate.php(153): OCA\PreviewGenerator\Command\Generate->parseFolder(Object(OC\Files\Node\Folder))
#7 /var/www/magicbroccoli.de/nextcloud/apps/previewgenerator/lib/Command/Generate.php(142): OCA\PreviewGenerator\Command\Generate->parseFolder(Object(OC\Files\Node\Folder))
#8 /var/www/magicbroccoli.de/nextcloud/apps/previewgenerator/lib/Command/Generate.php(96): OCA\PreviewGenerator\Command\Generate->generateUserPreviews(Object(OC\User\User))
#9 /var/www/magicbroccoli.de/nextcloud/lib/private/User/Manager.php(410): OCA\PreviewGenerator\Command\Generate->OCA\PreviewGenerator\Command\{closure}(Object(OC\User\User))
#10 /var/www/magicbroccoli.de/nextcloud/apps/previewgenerator/lib/Command/Generate.php(97): OC\User\Manager->callForSeenUsers(Object(Closure))
#11 /var/www/magicbroccoli.de/nextcloud/3rdparty/symfony/console/Command/Command.php(256): OCA\PreviewGenerator\Command\Generate->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#12 /var/www/magicbroccoli.de/nextcloud/3rdparty/symfony/console/Application.php(818): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#13 /var/www/magicbroccoli.de/nextcloud/3rdparty/symfony/console/Application.php(186): Symfony\Component\Console\Application->doRunCommand(Object(OCA\PreviewGenerator\Command\Generate), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#14 /var/www/magicbroccoli.de/nextcloud/3rdparty/symfony/console/Application.php(117): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#15 /var/www/magicbroccoli.de/nextcloud/lib/private/Console/Application.php(169): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
#16 /var/www/magicbroccoli.de/nextcloud/console.php(90): OC\Console\Application->run()
#17 /var/www/magicbroccoli.de/nextcloud/occ(11): require_once('/var/www/magicb...')
#18 {main}
I am using Nextcloud 11RC1 on a debian mashine with apache2 and php7.0.14 installed.
The process does not seem to be creating and storing previews for my images. At least when I open the Gallery app in Nextcloud it seems to have to generate ALL of the image previews again.
I get this message from cron for about 2 days now... every 15 minutes... i'm getting fat if I eat more cockies... :) what is causing this and how can I fix it?
Hello,
when I run:
/var/www/nextcloud# sudo -u www-data /usr/bin/php -f ./occ preview:"doesn't matter"
it goes thrue some image that generates error and script ends with log:
imagecreatefromjpeg(): gd-jpeg: JPEG library reports unrecoverable error: Unsupported marker type 0x28 at /var/www/nextcloud/lib/private/legacy/image.php#538
"0x28" suggest problem with file but:
I get this:
PHP Fatal error: Allowed memory size of 134217728 bytes exhausted (tried to allocate 5742 bytes) in /lib/private/legacy/image.php on line 538
not a big deal if a thumbmnail can't be generated, but I think it just kills the whole job, which should be avoided.
It's to be good to generate previews with custom resolution.
It's maybe something like that
occ preview:generate-all 200x100
syntax it's only example - need to be tuned.
Don't get me wrong: I was waiting for this app for a long time, so many thanks for making it real!
It is now time to ask for more 😉
In my use case, my server is a not-so-powerful Atom CPU that is not really able to generate a lot of previews at once in a reasonable time. However, I have my desktop that have full access to the source files (in my case, via NFS), so it can theoretically do the job in less time. Is this possible in the current design? Maybe the algorithm needs an access to the database?
This repository's website (as seen on the default GitHub view) should probably link to https://apps.nextcloud.com/apps/previewgenerator.
I'm recieving this error while generating a preview of an Android panorama image.
PHP Fatal error: imagecreatefromjpeg(): gd-jpeg: JPEG library reports unrecoverable error: Unsupported marker type 0x93 in /home/cloud/public_html/lib/private/legacy/image.php on line 538
I don't need to support this file types, but it would be great, if the generator creation will not be aborted on any kind of error.
@rullzer seems valuable for everyone, no? Or does it require some dependencies only available on certain systems, or admin interference?
There is no lockfile, flag or other control that indicates that preview generation is still running. Thus, when running ./occ preview:pre-generate
it will just start to do the same tasks. Depending on the activity on the instance, one run of ./occ preview:pre-generate
can already take a long time. So there is the chance that previews for each file are generated twice or multiple files (and making them processes run even longer).
The common thumbnail size for android is 128x128px, so this size should also be generated to have a real benefit on mobile clients.
@jancborchardt do you know which resolution is used by ios?
Hello Dev, ive encountered an Error which i cant solve on my own.
Im using Nextcloud 11.0.2 and php 7.0.8-0ubuntu0.16.04.3 with apache2 as the webserver.
When i start your tool from my terminal, it runs for a few seconds (no outputs are made to the console so im assuming it does at least something), and then it throws an exception. CPU usage is slightly higher than normal.
Thanks for your help, and if i can assist you further, feel free to ask!
Sorry for the missreferences, my log contained hashtags, which function i didnt knew...
The stacktrace:
An unhandled exception has been thrown:
Error: Call to a member function file_get_contents() on null in /var/www/mycloud/lib/private/Files/Filesystem.php:719
Stack trace:
0 /var/www/mycloud/lib/private/legacy/image.php(597): OC\Files\Filesystem::file_get_contents('/var/www/myclou...')
1 /var/www/mycloud/lib/private/Preview/MP3.php(80): OC_Image->loadFromFile('/var/www/myclou...')
2 /var/www/mycloud/lib/private/Preview/MP3.php(64): OC\Preview\MP3->getNoCoverThumbnail()
3 /var/www/mycloud/lib/private/Preview/GeneratorHelper.php(54): OC\Preview\MP3->getThumbnail('/files_trashbin...', 256, 256, false, Object(OC\Files\View))
4 /var/www/mycloud/lib/private/Preview/Generator.php(155): OC\Preview\GeneratorHelper->getThumbnail(Object(OC\Preview\MP3), Object(OC\Files\Node\File), 256, 256)
5 /var/www/mycloud/lib/private/Preview/Generator.php(108): OC\Preview\Generator->getMaxPreview(Object(OC\Files\SimpleFS\SimpleFolder), Object(OC\Files\Node\File), 'audio/mpeg')
6 /var/www/mycloud/lib/private/PreviewManager.php(182): OC\Preview\Generator->getPreview(Object(OC\Files\Node\File), 32, 32, true, 'fill', 'audio/mpeg')
7 /var/www/mycloud/apps/previewgenerator/lib/Command/PreGenerate.php(158): OC\PreviewManager->getPreview(Object(OC\Files\Node\File), 32, 32, true)
8 /var/www/mycloud/apps/previewgenerator/lib/Command/PreGenerate.php(181): OCA\PreviewGenerator\Command\PreGenerate->processFile(Object(OC\Files\Node\File))
9 /var/www/mycloud/apps/previewgenerator/lib/Command/PreGenerate.php(183): OCA\PreviewGenerator\Command\PreGenerate->processFolder(Object(OC\Files\Node\Folder))
10 /var/www/mycloud/apps/previewgenerator/lib/Command/PreGenerate.php(183): OCA\PreviewGenerator\Command\PreGenerate->processFolder(Object(OC\Files\Node\Folder))
11 /var/www/mycloud/apps/previewgenerator/lib/Command/PreGenerate.php(183): OCA\PreviewGenerator\Command\PreGenerate->processFolder(Object(OC\Files\Node\Folder))
12 /var/www/mycloud/apps/previewgenerator/lib/Command/PreGenerate.php(183): OCA\PreviewGenerator\Command\PreGenerate->processFolder(Object(OC\Files\Node\Folder))
13 /var/www/mycloud/apps/previewgenerator/lib/Command/PreGenerate.php(183): OCA\PreviewGenerator\Command\PreGenerate->processFolder(Object(OC\Files\Node\Folder))
14 /var/www/mycloud/apps/previewgenerator/lib/Command/PreGenerate.php(183): OCA\PreviewGenerator\Command\PreGenerate->processFolder(Object(OC\Files\Node\Folder))
15 /var/www/mycloud/apps/previewgenerator/lib/Command/PreGenerate.php(183): OCA\PreviewGenerator\Command\PreGenerate->processFolder(Object(OC\Files\Node\Folder))
16 /var/www/mycloud/apps/previewgenerator/lib/Command/PreGenerate.php(150): OCA\PreviewGenerator\Command\PreGenerate->processFolder(Object(OC\Files\Node\Folder))
17 /var/www/mycloud/apps/previewgenerator/lib/Command/PreGenerate.php(113): OCA\PreviewGenerator\Command\PreGenerate->processRow(Array)
18 /var/www/mycloud/apps/previewgenerator/lib/Command/PreGenerate.php(91): OCA\PreviewGenerator\Command\PreGenerate->startProcessing()
19 /var/www/mycloud/3rdparty/symfony/console/Command/Command.php(256): OCA\PreviewGenerator\Command\PreGenerate->execute(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
20 /var/www/mycloud/3rdparty/symfony/console/Application.php(818): Symfony\Component\Console\Command\Command->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
21 /var/www/mycloud/3rdparty/symfony/console/Application.php(186): Symfony\Component\Console\Application->doRunCommand(Object(OCA\PreviewGenerator\Command\PreGenerate), Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
22 /var/www/mycloud/3rdparty/symfony/console/Application.php(117): Symfony\Component\Console\Application->doRun(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
23 /var/www/mycloud/lib/private/Console/Application.php(169): Symfony\Component\Console\Application->run(Object(Symfony\Component\Console\Input\ArgvInput), Object(Symfony\Component\Console\Output\ConsoleOutput))
24 /var/www/mycloud/console.php(90): OC\Console\Application->run()
25 /var/www/mycloud/occ(11): require_once('/var/www/myclou...')
Hi,
I have a cron job with this script to run preview pre-generation twice a day, but since update it takes days to pre generate something, almost as long as full preview generation.
I have load around 5 new jpegs per day and 9653 at all.
Full preview generation takes around 3 days. But it is better, because it does not works before:)
Why it takes so long?
From the logs:
more nextcloud.log | grep Preview
{"app":"/var/www/nextcloud/occ preview:pre-generate","message":"+++ Preview generation is Completed. Time: 91595s +++","level":1,"time":"2017-03-20T02:26:36+01:00"}
{"app":"/var/www/nextcloud/occ preview:pre-generate","message":"+++ Starting Cron Preview generation +++","level":1,"time":"2017-03-20T13:00:02+01:00"}
{"app":"/var/www/nextcloud/occ preview:pre-generate","message":"+++ Preview generation is Completed. Time: 800s +++","level":1,"time":"2017-03-20T13:13:22+01:00"}
{"app":"/var/www/nextcloud/occ preview:pre-generate","message":"+++ Starting Cron Preview generation +++","level":1,"time":"2017-03-21T01:00:01+01:00"}
{"app":"/var/www/nextcloud/occ preview:pre-generate","message":"+++ Preview generation is Completed. Time: 106071s +++","level":1,"time":"2017-03-22T06:27:52+01:00"}
No warnings in terminal, no messages in nextcloud.log or Apache log.
Example of PDF file and more info:
https://help.nextcloud.com/t/preview-generator-1-0-1-runs-forever-app-is-not-usable-anymore/8481
Be sure to add './occ preview:pre-generate' to a cronjob
This project could use a change log.
Hey,
maybe a stupid question, but how can I add
To enable pre-generation of previews you must add ./occ preview:pre-generate to a system cron job
the generation of thumbnails, if I only can use the "Webcron" functionality of nextcloud.
Thanks for any hint
Hello,
how I can see if it works? When I do it manually (or with crontab) is there no output.
Edit1:
I tried it now with
sudo -u www-data php -f /var/www/occ preview:pre-generate -vvv
still no output. I'm doing something wrong? I'm using Ubuntu 16.04 with PHP7 with Nextcloud 11.
Greetings
Roger
Hi,
It would be much needed to generate only the sizes that are really needed.
I checked a random image and it had 21 previews. No wonder there is over 6 million previews on my servers, and my oc_filecache table is 3.2GB.
I browsed some random folders in the gallery app, and it looks like the gallery app only uses previews with heigths like 256, 512 and 2048. The files app uses 32x32 for the smallest preview and the sidebar preview was 768x512. At least some of the 21 generated previews is not ever needed.
Any thoughts?
The app stops working in case of mp3 without a cover image.
No way to debug it because the filename is truncated in nextcloud.log (see #19)
More info:
https://help.nextcloud.com/t/preview-generate-all-not-working/8190
Currently there is a timeout of 30min.
If, e.g. a preview generation fails with a fatal exception, the user would have to wait.
With a new option "--force" this timeout could be bypassed.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.