Code Monkey home page Code Monkey logo

geonames-mysql-dataimport's People

Contributors

bryant1410 avatar cauan avatar frost-nzcr4 avatar kroepke avatar maruhe avatar mrzard avatar nejo avatar rfay avatar roisholto avatar scmoral avatar talofo avatar vladrusu avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

geonames-mysql-dataimport's Issues

import-dump hangs

This is the output I get when importing the dumps

Importing geonames dumps into database geonames
mysql: [Warning] Using a password on the command line interface can be insecure.
########## Loading allCountries.txt... ##########
########## Loading allCountries.txt... ##########

The cursor blinks after the "Loading allCountries.txt" is shown twice, and that's it. Nothing else appears to be happening.
What is the expected output I should see after ./geonames_importer.sh -a import-dumps is run?
Should the process take minutes, or hours?

Implement 'data-directory' option

Hi again,

It would be great if we can specify a 'data-dir' (or somenthing like that) to keep downloaded files away from the files provieded in -tgz file.

Thanks in advance.

Project Website footer

The footer on the project site (says: "Hosted on GitHub Pages using the Dinky theme") is overlaying the bottom two buttons, so the links for TAR and GitHub aren't clickable.

Fails with latest Version of MySQL (8.0.12)

When run under the latest version of MySQL (8.0.12) (and perhaps others) import-dumps fails.

However, adding the following to the top of geonames_import_data.sql seems to fix it,
though I do not think this solution is good practice (with respect to security).

SET GLOBAL local_infile = 1;

...

MAMP import

/Applications/MAMP/Library/bin/ is needed in front of the mysql command in geonames_importer.sh or there should be instructions to make sure there is an alias to mysql that is set properly.

indexes in tables

I'm not one to bitch about be free code on github, but why they hell do you have indexes on every frkn column?

The Database must be in uft8mb4 instead utf8

All the data downloaded in the script is in utf8mb4 and the creation of the tables and columns must be in this codification . Replace all uftf8 with utf8mb4 and it will work.

Andorra missing

It's present in the data downloaded from geonames, importer doesn't import it.

Updating data

Hi guys. I'm trying to adapt this to update existing data.

Since the process is time expensive (downloading / importing), truncating and reimporting would result in DB downtime.

Geonames exposes their changes throught a modifications.txt file:
modifications-.txt : all records modified on the previous day, the date is in yyyy-MM-dd format. You can use this file to daily synchronize your own geonames database.

I guess we could run a a cronjob that fetches that file and runs the modifications.

Any idea how to tackle this?

Specifying collation order - the default ordering is not correct for non-English languages

The schema creation scripts correctly create the tables using the utf8 character set. However there is no specific collation order set, which means that collation would default to utf8_general_ci. However the general_ci incorrectly orders many non-English character sets - see https://stackoverflow.com/questions/766809/whats-the-difference-between-utf8-general-ci-and-utf8-unicode-ci for a discussion.

The solution is to declare a collation in the table DDL: COLLATE utf8_unicode_ci. With this collation, the ordering is correct.

Add -c option to wget command

Hi,

I't would be nice if you add '-c' option to wget command to ensure we only download the portion we don't already have in our local filesystem.

From 'man wget':
-c
--continue
Continue getting a partially-downloaded file. This is useful when you want to finish up a download started by a previous instance of Wget, or by another program.

Thanks in advance.

Keep the good job on this software.

Prefixes for table names

Hi!
I am proposing this instead of making a PR because I am not so good with shell programming to make the change.

But I'd be sweet to be able to add prefixes to tables. Just for grouping and keeping tables organized in a big legacy database.

geoname :Elevation can be ''

`elevation` int(11) DEFAULT NULL,

I just downloaded ES.txt and try to import it in a geoname table. There was many :"Incorrect integer value: '' for column 'elevation' at row 1 "
It seems at first glance that Either the database is corrupted, or the sql schema should accept a string. Or we should change the import to finetune that field and accept '' leads to NULL

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.