sfu-db / apiconnectors Goto Github PK

Website Description
An rapid API to get instagram data

Reason(s) to Support the Website
Instagram is a famous social APP which can be used to market, fetch useful information and so on.

Endpoints
https://instagram47.p.rapidapi.com/search/{search}
https://instagram47.p.rapidapi.com/get_user_id/{username}
https://instagram47.p.rapidapi.com/user_posts/{username}
https://instagram47.p.rapidapi.com/user_following/{userid}
https://instagram47.p.rapidapi.com/user_followers/{userid}
https://instagram47.p.rapidapi.com/public_user_posts/{user_id}
https://instagram47.p.rapidapi.com/post_comments/{post_id}
https://instagram47.p.rapidapi.com/location_search/{search}
https://instagram47.p.rapidapi.com/location_posts/{locationid}

Questions
What is the result when searching 'Kobe' in instagram?
What is the user id of username='stephencurry30'?
What are the posts for username='stephencurry30'?
Who did Curry follow?
Who followed Curry?

Website Description
It exposes city, region, and country data via both GraphQL and REST APIs.

Reason(s) to Support the Website
Collecting countries, cities information

Endpoints
...

Questions
Filter cities by name prefix, countries, location, time-zone, and minimum population
Get all country regions, states, and provinces
Get all cities in a given region
Get all countries supporting a currency
...

Website Description
OurAirport is a service that gives information on airports in certain regions and nearest airports.

Reason(s) to Support the Website
Developers making travel applications can use this website for accessing airport data based on certain properties.

Endpoints

• This API supports 6 endpoints but one gives 400 error and another endpoint serves the same purpose as one of the endpoints in IPLegit. A third requires one of two parameters and this API uses the template URL functionality which currently doesn't support this case.
  https://ourairport-data-search.p.rapidapi.com/airport
  https://ourairport-data-search.p.rapidapi.com/country
  https://ourairport-data-search.p.rapidapi.com/region

Questions

1. What is the country given GeoNames ID?
2. What are region names of a range of ID numbers?
3. What are all countries in Asia?

Describe the bug
First of all , this is not a critical issue.

Basically, while reviewing I faced the following issue. I saw that the developer was using a rudimentary form of getting the data, upon further examination, it was brought to my attention that system throws a assertion error when the config file is changed, as shown below.

To Reproduce
Steps to reproduce the behavior:

from dataprep.connector import connect
# You can get ”app_key“ by following https://www.themuse.com/developers/api/v2/apps
dc = connect('themuse', _auth={'access_token': app_key})
df = await dc.query('jobs', page=1, category='Data Science', location='Vancouver, Canada')
df[['id', 'name', 'company', 'locations', 'levels', 'publication_date']]

Desired behavior

Current behavior

Additional context
same issue found in airplanes api

Website 1 - Alpha Vantage, https://www.alphavantage.co/

• Describe the website
  Realtime and historical stock data
• Why it should be supported
  Realtime and historical stock data might be useful for investors economist, and data scientist.
• Endpoints - query
• Examples
  • What's the open price/ highest price/volume of IBM on 2021-1-29?
  • What is the daily/weekly/monthly historical stock data for IBM?

Website 2 - Holiday API, https://holidayapi.com/

• Describe the website
  Retrieves a list public holidays and observances for countries, states and provinces.
• Why it should be supported
  Holiday information may help improve products, marketing and scheduling.
• Endpoints - countries, holiday, workday
• Examples
  • What are the supported countries?
  • What are the holidays of Canada in 2020?
  • What are the previous / upcoming holidays of the specified data?
  • What the 100th workday starts from 2020-01-01?

• redesign: what content and how to show in the readme page
• editing the readme and implement the design

Website Description
Search for content within the iTunes Store and Apple Books Store.

Reason(s) to Support the Website
Developers will be able to search for a variety of content; including books, movies, podcasts, music, music videos, audiobooks, and TV shows.

Endpoints
https://itunes.apple.com/search?term=jack+johnson
https://itunes.apple.com/search?term=jack+johnson&entity=musicVideo
https://itunes.apple.com/search?term=jim+jones&country=ca
https://itunes.apple.com/lookup?id=909253&entity=album
https://itunes.apple.com/lookup?isbn=9780316069359

Questions

1. What is Jack Johnson's music videos?
2. What are all Jim Jones audio and video content? (only the results from the Canada iTunes Store)
3. Look up a book by its 13 digit ISBN.
4. Look up a Movie by AMG Video ID.
5. Look up an album or video by its UPC.

Website Description
The OMDB API is a RESTful web service to obtain movie information.

Reason(s) to Support the Website
Developers can use this data to make movie database websites and apps.

Endpoints
The API supports 1 endpoint but two sets of parameters to access information:

1. By title or ID
2. By search

Technically there is only 1 endpoint: http://www.omdbapi.com/, but I will make 2 config files - 1 for each set of parameters

Questions

1. List Avengers movies from most to least popular
2. What is the order of the following movies from highest to lowest amount of money made: Titanic, Avatar, Skyfall
3. Is "The Notebook" a sad or happy movie?

The image folder is superfluous and makes it look like a website name.
We should avoid putting it under the root folder and move it to other places appropriate.

There should be a tutorial on how to build their own config files for data connector.

Summary

Implement currents and coingecko API's configurationto DataConnectorConfigs

Currents

• Website:
  https://currentsapi.services/en

• Description of the websites:
  Currents api is an API for news data.

• Why to support this websites:
  This website can show us latest news, also can search news manually. I think some data scientists or students who want to do some research on news data will use this API

• Endpoints:

  • https://api.currentsapi.services/v1/search
  • https://api.currentsapi.services/v1/latest_news

• Questions:

  • How to get latest Chinese news?
  • How to get the political news about 'Trump'?
  • How to get the news about COVID-19 from 2020-12-25?

CoinGecko

• Website:
  https://www.coingecko.com

• Description of the websites:
  CoinGecko provides live pricing, trading volume, stocks, exchanges, historical data and other cryptocurrency data.

• Why to support this websites:
  User can get the live data and historical data of cryptocurrency. I think cryptocurrency investors or researchers will use this API.

• Endpoints:

  • https://api.coingecko.com/api/v3/coins/list
  • https://api.coingecko.com/api/v3/exchanges
  • https://api.coingecko.com/api/v3/search/trending
  • https://api.coingecko.com/api/v3/coins/markets
  • https://api.coingecko.com/api/v3/derivatives

• Questions:

  • What are the 10 cryptocurrencies with highest market cap and their current information?
  • What are the cryptocurrencies with highest increasing and decreasing percentage?
  • Which cryptocurrencies are trending in CoinGecko?
  • What are the 10 US exchanges with highest trade volume in the past 24 hours?
  • What are the 3 latest traded derivatives with perpetual contract?

Website Description
Provides data of Canada's economy, society and environment.

Reason(s) to Support the Website
Developers will be able to identify relationships between types of Canadian demographic data.

Endpoints
https://www12.statcan.gc.ca/rest/census-recensement/CR2016geo.json
https://www12.statcan.gc.ca/rest/census-recensement/CPR2016.json
https://www150.statcan.gc.ca/n1/dai-quo/ssi/homepage/ind-econ.json
https://www150.statcan.gc.ca/n1/dai-quo/ssi/homepage/ind-hp.json
https://www150.statcan.gc.ca/n1/dai-quo/ssi/homepage/schedule-key_indicators-eng.json

Questions

1. What is the age-sex distribution in the South Surrey-White Rock federal riding?
2. Which city in Canada has the highest percentage of Tamil-speaking people?
3. What types of post-secondary programs are most popular for each sex?
4. What is the relationship between language spoken and marital status?
5. Which city has the highest percentage of divorced couples?

Website Description
https://api.nasa.gov/
The objective of this site is to make NASA data, including imagery, eminently, etc.

Reason(s) to Support the Website
We haven't got an api for NASA data.

Endpoints
https://api.nasa.gov/planetary/apod
https://api.nasa.gov/DONKI/CME
https://api.nasa.gov/DONKI/GST
https://api.nasa.gov/DONKI/FLR
https://api.nasa.gov/DONKI/SEP

Questions
What are the title of Astronomy Picture of the Day from 2020-01-01 to 2020-01-10?
How to get Coronal Mass Ejection(CME) data from 2020-01-01 to 2020-02-01?
How many Geomagnetic Storms(GST) have occurred from 2020-01-01 to 2021-01-01? When is it?
How many Solar Flare(FLR) have occurred and completed from 2020-01-01 to 2021-01-01? How long did they last?
How to get Solar Energetic Particle(SEP) data from 2019-01-01 to 2021-01-01?

Website Description
Provides data of recent arrests, jail inmate search and mugshots.

Reason(s) to Support the Website
Developers can use this data to analyze criminal activity.

Endpoints

• This API supports 3 endpoints
  https://jailbase-jailbase.p.rapidapi.com/recent/
  https://jailbase-jailbase.p.rapidapi.com/search/
  https://jailbase-jailbase.p.rapidapi.com/sources/

Questions

1. How many Police Offices are in each US state?
2. Who were the 10 most recently arrested people by Wood County Sheriff's Department?
3. What is the URL for the mugshot of Almondo Smith?

The folder containing the images for the README was renamed without updating the README, resulting in the images not being visible in the document.

• Refine all the current config files
• Refine and add the ones in more_config branch

"Etsy" is the only website whose folder name is capitalized. Is it a bug?

Design and add a table to show the contributors of each configuration file in the readme page.
Please refer to this presentation here https://github.com/all-contributors/all-contributors#contributors-

Website Description
Covid Tracking is an Api that provides covid data about US nationally and by separate state.

Reason(s) to Support the Website
Developer using Covid data can easily use it to get up to date as well as historical data about Covid's impact

Endpoints

This API supports 6 endpoints
https://api.covidtracking.com/v2/states/{state}.json
http://api.covidtracking.com/v2/us/daily.json
http://api.covidtracking.com/v2/states.json
https://api.covidtracking.com/v2/us/daily/{date}.json
https://api.covidtracking.com/v2/states/{state}/daily.json
https://api.covidtracking.com/v2/states/{state}/{date}.json

Questions

What is the no. of cases in US?
What were the no. of cases in US on 2nd March?
What were the no. of cases in each state on 2nd March?
What is the no. of cases in New York?
What were the no. of cases in New York on 2nd March?

I hope to add some new APIs for this project but faced some problems during using the generator part. I have studied the guidance of adding APIs via making API describing configuration, while I'm more interested in the generator function mentioned in the paper.
I use the harvardartmuseum(HVM) API as a test for the generator.
As described in its API's documentation, the apikey is filled in Query Params. I successfully constructed my request and got correct response using Postman, while problems arose when I used the generator to send a sample request.
Initially, in the 3rd box(Authorization), I selected QueryParam as authentication option, and filled the key-value pair with apikey:content of my apikey. However, error message noted me that there is a KeyError in ui.py as the figure below.

I thought client_id and client_secret are not required for accessing HVM API, and this error seemed like a delete of an empty key in a dictionary, so I tried two methods to bypass this error.
First I tried to comment that part of the code locally, but the error still exists.

Second I tried to give these 3 keys a default value, but it arose more errors in pydantic package. I think these fields might be compulsory for this framework.
Moreover, if I tried to choose "No Authorization" option and write the query pair apikey:content of my apikey into the 2nd box. The generator normally return Unauthorized as I think.

So I want to ask how to fill the 3 fields in Authorization box: client_id, client_secret and access_token if I want to use generator to send sample requests to generate api configuration.

Website - Harvard Art Museums

• Describe the website
  The Harvard Art Museums API is a REST-style service designed for developers who wish to explore and integrate the museums’ collections in their projects. The API provides direct access to the data that powers the museums' website and many other aspects of the museums.
• Why it should be supported
  Collect information about Art
• Endpoints - objects, person, exhibition, publication, gallery
• Examples
  • Find all of the objects with the word "cat" in the title
  • Find all of the people affiliated with the museums
  • Find all of the exhibitions that take place at a Harvard Art Museums venue
  • Find all of the publications that were published in 2013
  • Find all of the galleries that are on floor (Level) 2 in the Harvard Art Museums building

Website Description
https://developers.amadeus.com/
Collect travel data, including airline, hotel, points of interest, activities.

Reason(s) to Support the Website
We haven't got an api for collecting travel data, which worth analyzing.

Endpoints
https://test.api.amadeus.com/v2/shopping/flight-offers
https://test.api.amadeus.com/v2/shopping/hotel-offers
https://test.api.amadeus.com/v1/shopping/activities
https://test.api.amadeus.com/v1/reference-data/locations/pois
https://test.api.amadeus.com/v1/safety/safety-rated-locations

Questions
What are the cheapest flights from Madrid to Paris on June 1st?
What are the best hotel deals during my trip?
What are the best tours and activities in this location?
How safe is this location?
What are the best places to visit in Barcelona?

Website Description
IPLegit is a service that gives information on IP addresses.

Reason(s) to Support the Website
This can be used to detect fraudulent IP addresses.

Endpoints

• This API supports 2 endpoints
  https://api.iplegit.com/full
  https://api.iplegit.com/info

Questions
Given a list of IP addresses from people who have visited a my website...

1. How can I check if someone's IP address is bad, so I can block them from my website? (will just write a print statement saying they have been blocked)
2. What country are most people from, who have visited my website?
3. Make a map showing locations of people who have visited my website.

Website Description
APIsguru is a service that gives information on web APIs.

Reason(s) to Support the Website
Developers can use this service to filter for APIs they need to perform certain tasks.

Endpoints
https://{region}.api.riotgames.com/lol/clash/v1/tournaments
https://{region}.api.riotgames.com/lor/ranked/v1/leaderboards
https://{region}.api.riotgames.com/tft/league/v1/master
https://{region}.api.riotgames.com/tft/league/v1/grandmaster
https://{region}.api.riotgames.com/tft/league/v1/challenger

Questions

1. How many days is the next League of Legends tournament running for?
2. Make tiers of League of Rinterra players based on their league points.
3. Who is the best veteran challenger?
4. List Grand Masters in order of win-loss ratio
5. List highest trending masters based on recent wins, activity and league points

Add python test code for config file schema correctness.
Every time a config file is generated, in the test case, the developer should call the test code and make sure there is no error.
Only when the test code is successful, the PR can be merged.
The test code should also be merged in the GUI back-end and run by default.

Website Description
Finds and verifies professional email addresses.

Reason(s) to Support the Website
Businesses can use this API to gather contact and professional information about people to send targeted advertisements to.

Endpoints
account
domain-search
email-count
email-verifier
email-finder

Questions

1. Who are executives of Intercom and what are their emails?
2. What is Dustin Moskovitz's email?
3. Are the emails of Stripe executives valid?
4. How many available requests do I have left?
5. What are the counts of each level of seniority of employees?

Website Description
TheSportsDB API is a open, crowd-sourced database of sports artwork and metadata with a free API.

Reason(s) to Support the Website
Developers for sports-related apps can use this API to support their products.

Endpoints
https://www.thesportsdb.com/api/v1/json/1/eventspastleague.php
https://www.thesportsdb.com/api/v1/json/1/searchteams.php
https://www.thesportsdb.com/api/v1/json/1/all_sports.php
https://www.thesportsdb.com/api/v1/json/1/lookup_all_teams.php
https://www.thesportsdb.com/api/v1/json/1/lookupteam.php

Questions

1. What were scores of the last 10 NBA games?
2. What is the oldest sports team in Toronto?
3. What are all team sports supported by TheSportsDB?
4. Which NBA stadium has highest seating capacity?
5. What are social media links of the Vancouver Canucks?

Hi, at present we need a query parameter to get twitter data. But many projects on twitter just ask for random tweets (for example get 1000 random tweets from twitter). Can we change q parameter to optional.