Code Monkey home page Code Monkey logo

saeedt / cfs_sampling Goto Github PK

View Code? Open in Web Editor NEW
4.0 5.0 1.0 37.91 MB

An optimal stratified sample design for Commodity Flow Survey (CFS) based on Simulated Annealing and Genetic Algorithm. A script in Procedural PostgreSQL is used to generate a frame with 100,000 records based on publicly available data.

License: MIT License

PLpgSQL 24.32% R 75.68%
commodity-flow-survey stratified-sampling genetic-algorithm simulated-annealing r sql

cfs_sampling's Introduction

CFS Sample Design Data and Scripts

Data and Scripts for the proposed sample design for CFS are stored in this repository. Following are the list of folders and their content.

Raw_Data

Main data sources used for generating the sample data are stored in this folder.

SQL

SQL Scripts used to create tables and anaylze the raw data are store in this folder. We used PostgreSQL which is a free open source database management system (DBMS). The queries and functions can be run on PostgreSQL 9.6 or later. Running on other SQL compatible DBMSs such as MySQL/MriaDB or MS SQL Server may require minor modifications.

  • SQL_Scripts.sql includes the scripts for creating tables and all queries developed for cleaning and aggregating the data. The comments in this file provide a high level explanation of each step. We used Common Table Expressions (CTEs) to merge multiple related queries in one step.
  • `Generate_est.sql' includes a function written in procedural PostgreSQL language that generates a sampling frame with user defined parameters based on CBP and FAF datasets.

Final_Data

Includes the final output of the scripts in SQL folder applied to the data in Raw_Data.

  • fafcbp.csv is the combined FAF and CBP datasets in CSV format. It is the disaggregated FAF data by county and NAICS based on CBP data. This data is needed by the generate_est function presented in SQL folder.
  • 100K_Frame_newCFS.csv is a set of 100,000 establishments generated with the generate_est function.

R_Scrripts

Includes the R scripts, functions used in the document.

cfs_sampling's People

Contributors

riesz-s avatar sa129715 avatar saeedt avatar zorost avatar

Stargazers

 avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar

Forkers

sahandsydney

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.