Comments (6)
stefina
commented on September 3, 2024
1
@davidread Thank you for fixing this! As I was on vacation I missed the notification about this. Sorry!
from ckanext-xloader.
davidread
commented on September 3, 2024
Thanks for the report. Are you able to create a failing test? This would be
a great help fixing it.
…On Sun, 19 May 2019 at 23:22, Nathan Perry ***@***.***> wrote:
On current master branch, when uploading a resource to datastore I get
the following error:
File "/vagrant_data/ckan/default/src/ckanext-xloader/ckanext/xloader/jobs.py", line 378, in set_resource_metadata
model.Resource.id == update_dict['resource_id']
KeyError: 'resource_id'
It appears that the set_resource_metadata function in
ckanext/xloader/jobs.py overwrites the update_dict dict:
update_dict = {'datastore_active': update_dict.get('datastore_active', True),
'datastore_contains_all_records_of_source_file': update_dict.get('datastore_contains_all_records_of_source_file', True)}
...so it no longer contains resource_id (see commit: 2a49a51
#diff-c548135301733293716337251017abb7
<2a49a51#diff-c548135301733293716337251017abb7>
)
However it tries to use update_dict['resource_id'] in the query below:
https://github.com/ckan/ckanext-xloader/blob/master/ckanext/xloader/jobs.py#L378
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#68?email_source=notifications&email_token=AACLDHF7PDYTJCZQYPAS433PWHHKBA5CNFSM4HN5QI72YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4GUTONZQ>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AACLDHDQFIF2RSJQC3G2QVTPWHHKBANCNFSM4HN5QI7Q>
.
from ckanext-xloader.
jeanpommier
commented on September 3, 2024
Hi,
The fix is quite easy:instead of replacing the dict, you just want to update it:
update_dict.update({'datastore_active': update_dict.get('datastore_active', True),
'datastore_contains_all_records_of_source_file': update_dict.get('datastore_contains_all_records_of_source_file', True)})
does the trick
from ckanext-xloader.
davidread
commented on September 3, 2024
@jeanpommier Thanks! A PR with test would be most welcome.
from ckanext-xloader.
jeanpommier
commented on September 3, 2024
Really ? For such a small fix, you need tests with it ? Well sorry but I won't have time right know, I've never written tests on ckan for now and it doesn't seem so easy to me. I can PR without tests for sure. If you need tests, well, it'll have to wait a bit...
from ckanext-xloader.
davidread
commented on September 3, 2024
The extra effort to include tests is much appreciated by the community.
Maybe @stefina can pitch in here, since this issue appears to relate to that work?
from ckanext-xloader.
Related Issues (20)
- xlsx import into datastore doesn't work
- Version is out-dated in setup.py HOT 1
- Response header location switches from https to http HOT 1
- Openpyxl cannot be imported
- No longer installable on Python 2.7 HOT 2
- Check datastore_active on resource update HOT 4
- Xloader with CKAN 2.9 error when upload resource xls
- Bad encoding for resources uploaded into DataStore
- Document download proxy config option
- Type guessing in 0.12 is less robust HOT 6
- How to mimic ckan.datapusher.callback_url_base
- resource_data page gives HTTP 504 timeout on very large resources
- File hash check doesn't take errors into account HOT 1
- README could mention that XLoader is capable of running on a separate server HOT 3
- encoding mix type issue, windows encoding not happy inside tabulator HOT 2
- Unterminated string literal HOT 3
- Bug: Race condition when datastore worker jobs processes large file in quick succession with active usage HOT 1
- Add synchronous CLI option HOT 6
- Update direct csv fast load to handle empty ending lines on csv and other edge conditions HOT 5
- Update enqueued job description/comment HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from ckanext-xloader.