Comments (10)
diff --git a/lib/docsplit/image_extractor.rb b/lib/docsplit/image_extractor.rb
index b7eed1c..09a4c86 100755
--- a/lib/docsplit/image_extractor.rb
+++ b/lib/docsplit/image_extractor.rb
@@ -28,7 +28,7 @@ module Docsplit
# Now we generate one page at a time, a counterintuitive opimization
# suggested by the GraphicsMagick list, that seems to work quite well.
def convert(pdf, size, format, previous=nil)
- tempdir = Dir.mktmpdir
+ tempdir = Dir.mktmpdir(nil,@tmpdir)
basename = File.basename(pdf, File.extname(pdf))
directory = directory_for(size)
pages = @pages || '1-' + Docsplit.extract_length(pdf).to_s
@@ -61,6 +61,7 @@ module Docsplit
@sizes = [options[:size]].flatten.compact
@sizes = [nil] if @sizes.empty?
@rolling = !!options[:rolling]
+ @tmpdir = options[:tmp]
end
# If there's only one size requested, generate the images directly into
diff --git a/lib/docsplit/text_extractor.rb b/lib/docsplit/text_extractor.rb
index 9ca017f..a250ff3 100644
--- a/lib/docsplit/text_extractor.rb
+++ b/lib/docsplit/text_extractor.rb
@@ -57,7 +57,7 @@ module Docsplit
# Extract a page range worth of text from a PDF via OCR.
def extract_from_ocr(pdf, pages)
- tempdir = Dir.mktmpdir
+ tempdir = Dir.mktmpdir(nil,@tmpdir)
base_path = File.join(@output, @pdf_name)
if pages
pages.each do |page|
@@ -119,6 +119,7 @@ module Docsplit
@force_ocr = options[:ocr] == true
@forbid_ocr = options[:ocr] == false
@clean_ocr = !(options[:clean] == false)
+ @tmpdir = options[:tmp]
end
end
from docsplit.
... and
diff --git a/lib/docsplit/command_line.rb b/lib/docsplit/command_line.rb
index d8290a7..a8ddbdd 100755
--- a/lib/docsplit/command_line.rb
+++ b/lib/docsplit/command_line.rb
@@ -73,6 +73,9 @@ Options:
def parse_options
@options = {:ocr => :default, :clean => true}
@option_parser = OptionParser.new do |opts|
-
opts.on('-t', '--tmpsubdir [DIR]', 'set the directory for all output') do |t|
-
@options[:tmp] = t
-
end
opts.on('-o', '--output [DIR]', 'set the directory for all output') do |d|
@options[:output] = d
end
from docsplit.
+1
from docsplit.
I was under the impression that Dir.mktmpdir
was supposed to be the reliable way to get a guaranteed temp dir across platforms. Where exactly is this failing?
from docsplit.
Dir.mktmpdir does provide a 'guaranteed' temporary directory... it's failing more from an operational aspect. If one is docsplit'ing a lot of documents, one will end up with a large number of docsplit directories.
For example, on OSX on my particular machine:
"/var/folders/8Q/8QdGGtLGHMa5PppDGWEyDE+++TI/-Tmp-/d20110913-9875-ehqpyp" is a typical directory created.
On a different unix derivative, they might accumulate in /tmp
Cleaning these up become an issue, when one is doing 10's or 100's of thousands of these a day, and from an operational aspect, removing them from an 'known' location (e.g. '/tmp/docsplit') is preferable:
Dir.mktmpdir(nil, '/tmp/delete-below-this')
=> "/tmp/delete-below-this/-p20110913-9875-ft6uez"
Any file older than an hour in the directory /tmp/delete-below-this is a candidate to be removed with extreme prejudice.
from docsplit.
Sorry, didn't mean to close the issue.
from docsplit.
In that case ... are you missing the ensure
clause at the bottom of those functions, that, well, ensures that the temp directories generated in this fashion are in fact cleaned up?
from docsplit.
Perhaps there's an issue there, since those temp files, over time, litter the /tmp directory. Thanks,
from docsplit.
Are you using the latest docsplit? I'm not seeing those directories.
from docsplit.
I hypothesize that as we've run it, on a background queue, on a busy machine running with the ruby processes monitored closely, if the ruby process is killed for some reason (OOM, etc.) then the ensure clause won't be executed, and the directories accumulate. We're running 0.50. Will try to schedule some time in the next week to look at this again.
from docsplit.
Related Issues (20)
- Percent sign in filenames isn't escaped properly
- "undefined method `strip' for nil:NilClass" occurs when attempting "Docsplit.extract_pdf" HOT 8
- encoding issue HOT 1
- rails invalid byte sequence in UTF-8 HOT 1
- Horizontal / table formatted text
- Executable filename issue with latest version (5.0.4) of LibreOffice on RHEL HOT 1
- Can any one please tell me how to pass file path as url to Docsplit ? HOT 2
- Docsplit::TextExtractor#extract_text should return the path of the output text file? HOT 2
- Downsampling has gotten worse in the last year
- Error "MAGICK_TEMPDIR" no se reconoce como comando interno o externo.
- Docsplit.extract_text auto orientation detection 'detect_orientation: true' param does not work.
- Email address contains more than three special chars(punctuation) is removed by Docsplit.clean_text method
- Different behavior on mac and linux
- diskspace leak when extracting text from pdf HOT 1
- Docsplit.extract_text generates a String with a null byte
- Docsplit.extract_images(path) => bin/rails: No such file or directory - file HOT 2
- Docsplit working on Dev, Staging server but not on Production.
- Docsplit::ExtractionFailed: gm convert: Unable to open file (/tmp/docsplit/58371.pdf) [No such file or directory]
- ruby 3.2 compatibility
- "Error: source file could not be loaded"
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from docsplit.