Comments (6)
Thank you for your report.
Could you possibly provide me the problematic JSON data?
I've tried to reproduce the problem with UTF-8 (without BOM) JSON file,
but I couldn't (sqlitebiter completed without errors).
from sqlitebiter.
I'm sorry that now I can't use my pc.
That json has many chinese words, you can write a simple json to test it, such as:
{
"sites": [
{ "name":"菜鸟教程" , "url":"m.runoob.com" },
{ "name":"google" , "url":"www.google.com" },
{ "name":"微博" , "url":"www.weibo.com" }
]
}
The reason may be sqlitebiter read the json as gbk codec, although that json is utf8.
from sqlitebiter.
The following method can get the original json:
- git clone this: https://github.com/Binaryify/NeteaseCloudMusicApi
- cd the path and node app.js (nodejs 6.0+)
- save the json from this address: http://localhost:3000/playlist/detail?id=612879229
Thank you for your reply.
from sqlitebiter.
debug info:
[DEBUG] sqlitebiter file: converting 'test.json'
[DEBUG] pytablereader: TableFileLoaderFactory.create_from_path: extension=json
[DEBUG] pytablereader: loading file: format=json, path=test.json, encoding=utf-8
[ERROR] sqlitebiter file: invalid json data format: path=test.json, message='gbk' codec can't decode byte 0xa6 in position 336: illegal multibyte sequence
[DEBUG] sqlitebiter file: ----- sqlitebiter completed -----
[DEBUG] sqlitebiter file: database path: test.sqlite
[DEBUG] sqlitebiter file: number of created table: 0
[DEBUG] sqlitebiter file:
[DEBUG] sqlitebiter file: ----- database schema -----
[DEBUG] SimpleSQLie: connect to a SQLite database: path='test.sqlite', mode=r
[DEBUG] SimpleSQLie: close connection to a SQLite database: path='G:\test.sqlite'
[DEBUG] sqlitebiter file:
[DEBUG] SimpleSQLie: close connection to a SQLite database: path='G:\test.sqlite'
original json:
test.zip
from sqlitebiter.
Thank you for your additional information.
I could reproduce the problem.
It looks like there are two problems, encoding and JSON format:
Problems
1. encoding
sqlitebiter
not properly opening for utf8 input files some formats (include JSON files).
This cause an encoding error at Windows environment.
2. JSON file format
Original JSON file (test.json
) that you mentioned in the previous comment is not convertible as it is by sqlitebiter
.
Convertible JSON formats for sqlitebiter
are very limited for now (details are described in here).
Solution
1. encoding
I've fix a dependency package that loading files.
Please try upgrade the sqlitebiter
dependency packages with pip install sqlitebiter --upgrade
.
You could convert utf8 files properly.
>type utf.json
{
"sites": [
{ "name":"闖憺ク滓蕗遞・ , "url":"m.runoob.com" },
{ "name":"google" , "url":"www.google.com" },
{ "name":"蠕ョ蜊・ , "url":"www.weibo.com" }
]
}
>sqlitebiter file utf.json
[INFO] sqlitebiter file: convert 'utf.json' to 'sites' table
>sqlite3 out.sqlite
sqlite> .schema
CREATE TABLE IF NOT EXISTS 'sites' (name TEXT, url TEXT);
2. JSON file format
You could extract convertible JSON data by jq command from the JSON, and convert it by sqlitebiter
.
If you have a certain database schema that you would like to extract from the JSON file.
from sqlitebiter.
Thank you for your reply!
from sqlitebiter.
Related Issues (20)
- update $archive value to "sqlitebiter_windows_amd64.zip" in get-sqlitebiter.ps1 HOT 1
- How do I specify columns orders when loading form an array of JSON objects? HOT 3
- Feature request: input from stdout HOT 1
- SQL table name cannot start with numeric? HOT 1
- Getting "codec can't decode byte ..." HOT 2
- json to sqlLite Error HOT 1
- ODS Support
- Convert excel DATE to sqlite DATE
- Unable to locate package HOT 1
- Retain leading zeros when converting text or --no-type-inference HOT 1
- Support simple hierarchical JSON HOT 2
- Automatically detect and create Primary and Foreign Keys HOT 2
- Better default name for database file
- Non able to create output, starting from a simple html page HOT 4
- incompatibility between pip V22 json output and sqlitebiter HOT 1
- Installation dpkg (.deb package) Failing HOT 1
- Convert to spaces HOT 2
- Add --unescape option HOT 1
- Allow inferring source format from alternate string HOT 6
- sqlite3.IntegrityError: NOT NULL constraint failed: _source_info_.format_name HOT 2
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from sqlitebiter.