Code Monkey home page Code Monkey logo

pysocialwatcher's Introduction

pySocialWatcher

PRs Welcome

A Social Data Collector from Facebook Marketing API

I'm more than pleased that many teams used this library. But I need your help to support it. Please feel free to pull requests that solve issues. Unfortunatelly, my time is very very limited to keep updating this repository.

If this package helps your research somehow, reference this paper:

@inproceedings{araujo2017facebook,
 author = {Araujo, Matheus and Mejova, Yelena and Weber, Ingmar and Benevenuto, Fabricio},
 title = {Using Facebook Ads Audiences for Global Lifestyle Disease Surveillance: Promises and Limitations},
 series = {WebSci '17},
 year = {2017},
 location = {Troy, USA},
 publisher = {ACM},
 address = {New York, NY, USA},
 keywords = {Facebook, Advertising, Epidemihology, Social Media, Health},
} 

Build Status codecov

Package Name: pysocialwatcher

Facebook Ads API version supported: 2.9

License: MIT

Python Version: 2.7

What is this for

This package tries to get the full potencial of the Facebook Marketing API for Social Analysis research. Recent works show that online social media has a huge potencial to provide interesting insights on trends of across demographic groups.

Examples of research question that it can answer:

  • For each european country, get how many people are interested in Science?
  • Get how many people in each GCC country who is Graduated AND is interested in Football, and how many is not interested in Football breakdown by: gender, age range, scholarity, language and citizenship.
Tutorial

A guide for getting started with Facebook Ads API data collection can be found here: https://worldbank.github.io/connectivity_mapping/intro.html Please note that you will need to clone a fork (https://github.com/joaopalotti/pySocialWatcher), instead of the original repo. The fork includes support for Python 3 (Python 3.7 works for sure, very likely it should support Python 3.8 as well) and utilities to create JSON files and to post-process your collection.

Facebook Marketing API References page:

Targeting Specs: https://developers.facebook.com/docs/marketing-api/targeting-specs/v2.8

Ad Targeting Search API: https://developers.facebook.com/docs/marketing-api/targeting-search/v2.8

Limitations:

  • Current supported API fields are listed below:
    "interests",
    "behaviors",
    "education_statuses",
    "family_statuses",
    "relationship_statuses",
    "locales",
    "genders",
    "age_min",
    "age_max",
    "geo_locations"
    

Install

git clone https://github.com/maraujo/pySocialWatcher.git
cd pySocialWatcher
pip install -r requirements.txt
python setup.py install

Quick Start

You should have a .csv file with your Facebook tokens and accountIDs. Example: pySocialWatcher/pysocialwatcher/facebook_tokens_example.csv

>>> from pysocialwatcher import watcherAPI 
>>> watcher = watcherAPI() 
>>> watcher.load_credentials_file("pysocialwatcher/credentials.csv")
>>> watcher.run_data_collection("pysocialwatcher/input_examples/quick_example.json")

How it works (slides):

Check the slides: https://goo.gl/WzE9ic

Features

  1. Static input json format to make you experiments easily reproducible.
  2. Support multiple Facebook tokens.
  3. Multiple tokens are processed in parallel to speedup data collection.
  4. Complex logic queries in the Facebook Marketing API with 'or', 'and', 'not', for example:.
      "interests": [{
            "not": [6003442346642],
            "and": [6004115167424, 6003277229371],
            "name": "Not interested in Football, but interest in some physical activity"
        }
  1. Automatically save the state every constants.SAVE_EVERY requests. If any problem happens you can load the incomplete file and continue the data collection (load_data_and_continue_collection)

Input Json Format Example

The following input is an example of input format of the package. In this example it will perform several requests in the Facebook Marketing API in order to collect the audience realted to Soccer Interest in GCC countries.

{   "name": "Soccer Interest",
     "geo_locations": [
          { "name": "countries", "values": ["BH"] },
          { "name": "countries", "values": ["KW"] },
          { "name": "countries", "values": ["OM"] },
          { "name": "countries", "values": ["QA"] },
          { "name": "countries", "values": ["SA"] },
          { "name": "countries", "values": ["AE"] }
],
"genders": [1,2],
"ages_ranges": [
    {"min":18, "max":24},
    {"min":55}
],
"scholarities":[{
    "name" : "Graduated",
    "or" : [3,7,8,9,11]
  }
],
"languages":[{
    "name" : "Arabic",
    "values" : [28]
    },
    null
],
"behavior": [
    {
      "or": [6015559470583],
      "name": "Expats"
    },
    {
      "not": [6015559470583],
      "name": "Not Expats"
    }
],
"interests": [{
        "not": [6003442346642],
        "and": [6004115167424, 6003277229371],
        "name": "Not interested in Football, but interest in physical activity"
    },{
        "or": [6003442346642],
        "name" : "Football"
    }
]
}

In total it will perform 192 requests which are created in the following way:

For each GCC country in the geo_locations field:
    For each gender in [1,2]:
        For each age range in [18-24,55+]:
            For each scholarity group:
                For each language group:
                    For each behavior group:
                        For each interest group:
                            doFacebookAPIRequest()
            

So it will collect the audience for all of the combinations specified in the input file. If you don't want to specify a specific field you can ommit in the input or put a null value in the list like:

"languages":[{
    "name" : "Arabic",
    "values" : [28]
    },
    null
],

Find Targeting IDs given Query

>>> from pysocialwatcher import watcherAPI 
>>> watcher = watcherAPI() 
>>> watcher.load_credentials_file("pysocialwatcher/credentials.csv")
>>> watcher.print_search_targeting_from_query_dataframe("Parents")

+----+-----------------+----------------------------------------+------------------+------------------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------+
|    |   audience_size | description                            |               id | name                                                 | path                                                                                                    | type              |
|----+-----------------+----------------------------------------+------------------+------------------------------------------------------+---------------------------------------------------------------------------------------------------------+-------------------|
|  0 |       286670228 | People who are parents.                |    6002714398372 | Parents (All)                                        | [u'Demographics', u'Parents', u'All Parents', u'Parents (All)']                                         | family_statuses   |
|  1 |        92661179 | Parents with children 18-26 years old. |    6023005718983 | (18-26 Years) Parents with Adult Children            | [u'Demographics', u'Parents', u'All Parents', u'(18-26 Years) Parents with Adult Children ']            | family_statuses   |
|  2 |        45169601 | Parents with children 13-18 years old. |    6023005681983 | (13-18 Years) Parents with Teenagers                 | [u'Demographics', u'Parents', u'All Parents', u'(13-18 Years) Parents with Teenagers ']                 | family_statuses   |
|  3 |        13123437 | Parents with Children ages 8-12        |    6023080302983 | (08-12 Years) Parents with Preteens                  | [u'Demographics', u'Parents', u'All Parents', u'(08-12 Years) Parents with Preteens']
...

Find Interest IDs given name Name

>>> from pysocialwatcher import watcherAPI 
>>> watcher = watcherAPI() 
>>> watcher.load_credentials_file("pysocialwatcher/credentials.csv")
>>> watcher.print_interests_given_query("Family")

+----+------------+---------------+--------------------------+--------------------------------------------------------------------+
|    |   audience |   interest_id | name                     | path                                                               |
|----+------------+---------------+--------------------------+--------------------------------------------------------------------|
|  0 | 1019524230 | 6012684376438 | Family and relationships | [u'Interests', u'Family and relationships']                        |
|  1 |  788072890 | 6003476182657 | Family                   | [u'Interests', u'Family and relationships', u'Family']             |
|  2 |  178716020 | 6003190413105 | Family (biology)         | [u'Interests', u'Additional Interests', u'Family (biology)']       |
|  3 |   58127760 | 6003206382686 | family  planning         | [u'Interests', u'Additional Interests', u'family  planning']       |
|  4 |   25284610 | 6002966041646 | family films             | [u'Interests', u'Additional Interests', u'family films']           |
|  5 |   18734150 | 6003305411169 | Family Guy               | [u'Interests', u'Additional Interests', u'Family Guy']             |
|  6 |   12447740 | 6003455599405 | Royal family             | [u'Interests', u'Additional Interests', u'Royal family']           |
|  7 |   10252970 | 6003143952966 | Family of Barack Obama   | [u'Interests', u'Additional Interests', u'Family of Barack Obama'] |
+----+------------+---------------+--------------------------+--------------------------------------------------------------------+

Find Behavior IDs Lists

>>> from pysocialwatcher import watcherAPI 
>>> watcher = watcherAPI() 
>>> watcher.load_credentials_file("pysocialwatcher/credentials.csv")
>>> watcher.print_behaviors_list()
+-----+------------+---------------+-------------------------------------------------------------------------------------------------------------------+----------------------------------------------------+--------------------------------------------------------------------------------------------+
|     |   audience |   behavior_id | description                                                                                                       | name                                               | path                                                                                       |
|-----+------------+---------------+-------------------------------------------------------------------------------------------------------------------+----------------------------------------------------+--------------------------------------------------------------------------------------------|
|   0 |  249325232 | 6002714895372 | People whose activities on Facebook suggest they are Frequent travelers.                                          | All frequent travelers                             | [u'Travel', u'All frequent travelers']                                                     |
|   1 |   34932108 | 6002714898572 | People who list themselves as small business owners or own small business pages on Facebook                       | Small business owners                              | [u'Digital activities', u'Small business owners']                                          |
|   2 |    6444115 | 6002764392172 | People who have used Facebook Payments platform in the past 90 days                                               | FB Payments (All)                                  | [u'Digital activities', u'FB Payments (All)']                                              |
|   3 |   91484312 | 6003050295572 | Users who uploaded >50 photos on Facebook in last month.                                                          | Photo uploaders
...

Find Geo Location Key Given Query and Location Type

>>> from pysocialwatcher import watcherAPI 
>>> watcher = watcherAPI() 
>>> watcher.load_credentials_file("pysocialwatcher/credentials.csv")
>>> watcherAPI.print_geo_locations_given_query_and_location_type("new", ["city"])
+----+---------+-------------------------+-----------------+-------------------+--------+
|    |     key | name                    |   supports_city |   supports_region | type   |
|----+---------+-------------------------+-----------------+-------------------+--------|
|  0 | 2490299 | New York                |               1 |                 1 | city   |
|  1 | 2490287 | New Rochelle            |               1 |                 1 | city   |
|  2 | 2528778 | New Braunfels           |               1 |                 1 | city   |
|  3 | 2511352 | New Castle              |               1 |                 1 | city   |
....

Find all Geo Locations Given Location Type and Country

>>> from pysocialwatcher import watcherAPI 
>>> watcher = watcherAPI() 
>>> watcher.load_credentials_file("pysocialwatcher/credentials.csv")
>>> watcherAPI.print_geo_locations_given_query_and_location_type(None, ["region"], country_code='ID')

+----+----------------+----------------+-------+------------------------------+-----------------+-------------------+--------+
|    | country_code   | country_name   |   key | name                         | supports_city   | supports_region   | type   |
|----+----------------+----------------+-------+------------------------------+-----------------+-------------------+--------|
|  0 | ID             | Indonesia      |  1662 | Bali                         | True            | True              | region |
|  1 | ID             | Indonesia      |  1676 | East Nusa Tenggara           | True            | True              | region |
|  2 | ID             | Indonesia      |  1685 | West Java                    | True            | True              | region |
|  3 | ID             | Indonesia      |  1675 | West Nusa Tenggara           | True            | True              | region |
...

Find KML Given Location Key and Location Type

>>> from pysocialwatcher import watcherAPI 
>>> watcher = watcherAPI() 
>>> watcher.load_credentials_file("pysocialwatcher/credentials.csv")
>>> watcherAPI.get_kml_given_geolocation("countries", ["BR","CL","AT","US","QA"])
+----+---------------------------------------------------+-------------------+
|    |     kml                                           | name              |
|----+---------------------------------------------------+-------------------+
|  0 | <Polygon><outerBoundaryIs><LinearRing><coordin... | Brazil            |
|  1 | <Polygon><outerBoundaryIs><LinearRing><coordin... | Chile             |
|  2 | <Polygon><outerBoundaryIs><LinearRing><coordin... | Austria           |
|  3 | <Polygon><outerBoundaryIs><LinearRing><coordin... | United States     |
....

Advaced Configurations

Change Sleep time between requests to 10s:
from pysocialwatcher import constants
constants.SLEEP_TIME = 10
Change save temporary file every 1000 requests:
from pysocialwatcher import constants
constants.SAVE_EVERY = 1000
Specify that results should be saved to the directory data/pySocialWatcher/test_query_results/
watcher.run_data_collection("pySocialWatcher/pysocialwatcher/quick_example.json", 
                            "data/pySocialWatcher/test_query_results/")

Note: Assumes that the directory already exists. Filepath is added to the beginning of the output .csv files generated.

Potential Issues:

  1. If you received the error: You are calling a deprecated version of the Ads API, means that Facebook updated the API. One way to fix is changing the first 3 variables of the constants.py file to the current Facebook API. This does not guarantee that everything will work.
  2. Various users are getting issues with this package. Since I personally do not use it more, I recommend the fork from Joao Palotti.

Change Log

  • 2.0a - Fix bug from @joaopalotti and thank @KangboLu, @kpolimis, @khof312, @mfatehkia for previous commits.
  • 0.1j - Get more informative dataframes from: get_behavior_dataframe, get_interests_given_query, get_search_targeting_from_query_dataframe
  • 0.1i - Errors more understable and some small fixes.
  • 0.1h - AND working for behavior. Thanks @ianbstewart.
  • 0.1g - Besides MAU, now we also captures DAU. Thanks @VatsalaSingh.
  • 0.1f - Call version v2.11 of Facebook Marketing API
  • 0.1e - Add support to not for demographics that uses specific integer numbers such as scholarity
  • 0.1d - Add support to household_composition
  • 0.1b2 - Add support to relationship_statuses and check input keys
  • 0.1b0 - Search any query with get_search_targeting_from_query_dataframe()
  • 0.1a3 - Resilient error handling when error code = 2 (try again more constants.MAX_NUMBER_TRY times)
  • 0.1a2 - Fix bug when have multiple operators like (and, not) in the same query
  • 0.1a1 - Add MIT License and 'Geo Location' field support in facebook API
  • 0.1a0 - First Alpha Release

pysocialwatcher's People

Contributors

dependabot[bot] avatar ianbstewart avatar joaopalotti avatar kangbolu avatar khof312 avatar kpolimis avatar maraujo avatar mfatehkia avatar valdrst avatar vatsalasingh avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

pysocialwatcher's Issues

"Update the value of the location_types field"

Hi,

first, thanks for creating this package!

I was just trying to get the MWE from the World Bank to run (https://worldbank.github.io/connectivity_mapping/intro.html), and immediately ran into this error message:

'{"error":{"message":"Invalid parameter","type":"OAuthException","code":100,"error_data":{"blame_field_specs":[["targeting"]]},"error_subcode":1870199,"is_transient":false,"error_user_title":"Update the value of the location_types field","error_user_msg":"All location targeting will now reach people living in or recently in the locations that you selected. Please remove all values from the location_types field.","fbtrace_id":"XXXXXXXX"}}'

It seems Facebook has changed how people's locations are targeted: https://www.facebook.com/business/help/365561350785642?id=176276233019487

I'd fix it myself, but am pretty much a newbie when it comes to Python. Do you maybe have a fix for this?

Thanks again in any case,
Carlo

MINOR ISSUE: Innacurate documentation of SLEEP constant

Hey I was trying to fiddle with the SLEEP constant as per the documentation below:

"Advaced Configurations
#####Change Sleep time between requests to 10s:

from pysocialwatcher import constants
constants.SLEEP = 10"

but the correct constant seems to be 'SLEEP_TIME' rather than 'SLEEP'

Best,

Julian

Requests greater than 300

Seems like when I try to make requests greater than 300 the progress halts indefinitely (with two accounts in credentials.csv). How do I make sure I continue to get data?

"Param currency on field delivery_estimate" Error

Hi,
i've been trying to collect some data from facebook in order to learn how to use pySocialWatcher. I've setup an account with ID and Token, followed your instructions and typed command:

watcher.run_data_collection("pysocialwatcher/input_examples/jdafermos_example.json")

where my "jdafermos_example.json" file is:

{ "name": "Data in Crete",
"geo_locations": [
{ "name": "regions", "values": [{"key":"4170"}], "location_types": ["region"] }
],
"genders": [0],
"ages_ranges": [
{"min":18, "max":75}
]
}

The skeleton file is saved, but when it tries to collect the data an error message about "Param currency on field delivery_estimate" appears which includes the following lines:

FatalException: '{"error":{"message":"(#100) Param currency on field delivery_estimate: This param is not valid","type":"OAuthException","code":100,"fbtrace_id":"Ahe7jEVHUwoWhbm2wUTnMVQ"}}'

pysocialwatcher.utils.FatalException: 'FatalError: Check logging for clue. No way to proceed from here.'

Do you know if there is any solution to this?
Thank you

Response doesn't return JSON

I tried executing a basic query for NY ex-pats (JSON below) using utils.process_audience_from_response but the query was unable to complete because the response did not contain a JSON. Do you know if the API has changed and is now returning non-JSON responses?

query:

{   "name": "Expats Distribution in New York",
    "geo_locations": [
      { 
        "name": "cities",
        "values": [{"key":"2490299"}],
        "location_types": ["home"] 
      }
    ],
    "behavior": [
        {"or" : [6019673233983], "name" : "Expats (Zimbabwe)"},
        {"or" : [6027149006383], "name" : "Expats (Vietnam)"},
        {"or" : [6026404871583], "name" : "Expats (Venezuela)"},
        {"or" : [6023620475783], "name" : "Expats (US)"},
        {"or" : [6019396649183], "name" : "Expats (United States)"},
        {"or" : [6021354152983], "name" : "Expats (UK)"},
        {"or" : [6019673501783], "name" : "Expats (Uganda)"},
        {"or" : [6023516430783], "name" : "Expats (UAE)"},
        {"or" : [6019366994183], "name" : "Expats (Turkey)"},
        {"or" : [6019377644783], "name" : "Expats (Switzerland)"},
        {"or" : [6023516315983], "name" : "Expats (Sri Lanka)"},
        {"or" : [6019366943583], "name" : "Expats (Spain)"},
        {"or" : [6027148973583], "name" : "Expats (South Korea)"},
        {"or" : [6019564383383], "name" : "Expats (South Africa)"},
        {"or" : [6023516403783], "name" : "Expats (Singapore)"},
        {"or" : [6027149004983], "name" : "Expats (Serbia)"},
        {"or" : [6023357000583], "name" : "Expats (Senegal)"},
        {"or" : [6025000813183], "name" : "Expats (Saudi Arabia)"},
        {"or" : [6025670492783], "name" : "Expats (Rwanda)"},
        {"or" : [6025000815983], "name" : "Expats (Russia)"},
        {"or" : [6027148962983], "name" : "Expats (Romania)"},
        {"or" : [6019520122583], "name" : "Expats (Puerto Rico)"},
        {"or" : [6021354882783], "name" : "Expats (Portugal)"},
        {"or" : [6019396657183], "name" : "Expats (Poland)"},
        {"or" : [6018797091183], "name" : "Expats (Philippines)"},
        {"or" : [6027149008183], "name" : "Expats (Peru)"},
        {"or" : [6016916394383], "name" : "Expats (Pakistani)"},
        {"or" : [6027147189983], "name" : "Expats (Pakistan)"},
        {"or" : [6018797004183], "name" : "Expats (Nigeria)"},
        {"or" : [6023516368383], "name" : "Expats (New Zealand)"},
        {"or" : [6023287393783], "name" : "Expats (Netherlands)"},
        {"or" : [6023356955383], "name" : "Expats (Nepal)"},
        {"or" : [6023516338783], "name" : "Expats (Morocco)"},
        {"or" : [6023676072183], "name" : "Expats (Mexico)"},
        {"or" : [6027147160983], "name" : "Expats (Malaysia)"},
        {"or" : [6068613839383], "name" : "Expats (Latvia)"},
        {"or" : [6018796980983], "name" : "Expats (Kenya)"},
        {"or" : [6023676028783], "name" : "Expats (Japan)"},
        {"or" : [6019396654583], "name" : "Expats (Italy)"},
        {"or" : [6025000823583], "name" : "Expats (Israel)"},
        {"or" : [6019396650783], "name" : "Expats (Ireland)"},
        {"or" : [6027149017383], "name" : "Expats (Iran)"},
        {"or" : [6019564344583], "name" : "Expats (Indonesia)"},
        {"or" : [6016916298983], "name" : "Expats (India)"},
        {"or" : [6019396638383], "name" : "Expats (Hungary)"},
        {"or" : [6023676022783], "name" : "Expats (Hong Kong)"},
        {"or" : [6059793664583], "name" : "Expats (Honduras)"},
        {"or" : [6018797373783], "name" : "Expats (Haiti)"},
        {"or" : [6019673808383], "name" : "Expats (Guatemala)"},
        {"or" : [6023676017583], "name" : "Expats (Greece)"},
        {"or" : [6019673448383], "name" : "Expats (Ghana)"},
        {"or" : [6019367052983], "name" : "Expats (Germany)"},
        {"or" : [6019367014383], "name" : "Expats (France)"},
        {"or" : [6068209522983], "name" : "Expats (Finland)"},
        {"or" : [6018797165983], "name" : "Expats (Ethiopia)"},
        {"or" : [6023287351383], "name" : "Expats (Estonian)"},
        {"or" : [6019673777983], "name" : "Expats (El Salvador)"},
        {"or" : [6027148953383], "name" : "Expats (Egypt)"},
        {"or" : [6019673762183], "name" : "Expats (Dominican Republic)"},
        {"or" : [6018797127383], "name" : "Expats (Cuba)"},
        {"or" : [6019673525983], "name" : "Expats (Colombia)"},
        {"or" : [6019452369983], "name" : "Expats (China)"},
        {"or" : [6025054896983], "name" : "Expats (Chile)"},
        {"or" : [6019396764183], "name" : "Expats (Canada)"},
        {"or" : [6018797036783], "name" : "Expats (Cameroon)"},
        {"or" : [6019564340583], "name" : "Expats (Brazil)"},
        {"or" : [6043702804583], "name" : "Expats (Belgium)"},
        {"or" : [6023356562783], "name" : "Expats (Bangladesh)"},
        {"or" : [6023675997383], "name" : "Expats (Austria)"},
        {"or" : [6021354857783], "name" : "Expats (Australia)"},
        {"or" : [6025000826583], "name" : "Expats (Argentina)"}
    ],
    "ages_ranges": [
        {"min":18}
    ],
    "genders": [0]
}

error:

ValueErrorTraceback (most recent call last)
<ipython-input-24-33f40b778386> in <module>()
     25 
     26 watcher.load_credentials_direct(FB_KEY,FB_APPLICATION_ID)
---> 27 watcher.run_data_collection(SETTINGS_FILE_NAME)
     28 files.download('results.csv')

/usr/local/lib/python2.7/dist-packages/pysocialwatcher/main.pyc in run_data_collection(json_input_file_path)
    177         PySocialWatcher.check_input_integrity(input_data_json)
    178         collection_dataframe = PySocialWatcher.build_collection_dataframe(input_data_json)
--> 179         collection_dataframe = PySocialWatcher.perform_collection_data_on_facebook(collection_dataframe)
    180         return collection_dataframe
    181 

/usr/local/lib/python2.7/dist-packages/pysocialwatcher/main.pyc in perform_collection_data_on_facebook(collection_dataframe)
    142         print_info("Data Collection Complete")
    143         save_temporary_dataframe(collection_dataframe)
--> 144         post_process_collection(collection_dataframe)
    145         save_after_collecting_dataframe(collection_dataframe)
    146         return collection_dataframe

/usr/local/lib/python2.7/dist-packages/pysocialwatcher/utils.pyc in post_process_collection(collection_dataframe)
    327     print_info("Computing Audience column")
    328     collection_dataframe["audience"] = collection_dataframe["response"].apply(
--> 329         lambda x: process_audience_from_response(x))
    330     return collection_dataframe
    331 

/usr/local/lib/python2.7/dist-packages/pandas/core/series.pyc in apply(self, func, convert_dtype, args, **kwds)
   2549         index_col : int or sequence, default 0
   2550             Column to use for index. If a sequence is given, a MultiIndex
-> 2551             is used. Different default from read_table
   2552         encoding : string, optional
   2553             a string representing the encoding to use if the contents are

pandas/_libs/src/inference.pyx in pandas._libs.lib.map_infer()

/usr/local/lib/python2.7/dist-packages/pysocialwatcher/utils.pyc in <lambda>(x)
    327     print_info("Computing Audience column")
    328     collection_dataframe["audience"] = collection_dataframe["response"].apply(
--> 329         lambda x: process_audience_from_response(x))
    330     return collection_dataframe
    331 

/usr/local/lib/python2.7/dist-packages/pysocialwatcher/utils.pyc in process_audience_from_response(literal_response)
    319 
    320 def process_audience_from_response(literal_response):
--> 321     audience = json.loads(literal_response)["data"]["users"]
    322     return int(audience)
    323 

/usr/lib/python2.7/json/__init__.pyc in loads(s, encoding, cls, object_hook, parse_float, parse_int, parse_constant, object_pairs_hook, **kw)
    337             parse_int is None and parse_float is None and
    338             parse_constant is None and object_pairs_hook is None and not kw):
--> 339         return _default_decoder.decode(s)
    340     if cls is None:
    341         cls = JSONDecoder

/usr/lib/python2.7/json/decoder.pyc in decode(self, s, _w)
    362 
    363         """
--> 364         obj, end = self.raw_decode(s, idx=_w(s, 0).end())
    365         end = _w(s, end).end()
    366         if end != len(s):

/usr/lib/python2.7/json/decoder.pyc in raw_decode(self, s, idx)
    380             obj, end = self.scan_once(s, idx)
    381         except StopIteration:
--> 382             raise ValueError("No JSON object could be decoded")
    383         return obj, end

ValueError: No JSON object could be decoded

(noob q): unable to install

Apologies if this is a very basic question, but I'm still learning my ropes with Python, and wanted to use this project. I carried out the installation instructions, and then initiated jupyter lab running a python 2 kernel to run the code. However, when I try to run watcherAPI, I keep running into the same error- "No Module named main". Any help would be greatly appreciated, I haven't been able to find any tutorial notebooks either, but I'm happy to write one once I'm able to do this.

screenshot

Deprecated API?

I had to update REACHESTIMATE_URL in constants.py because one of the tokens that I was using was not working with the old API (recently updated from 2.11 to 3.0).

REACHESTIMATE_URL = "https://graph.facebook.com/v3.0/act_{}/delivery_estimate"

ads API v8.0

Facebook updated their ads API to v8.0 in August of this year. Have you managed to make pySocialWatcher run with these changes? Changing the 3 constants as you mentioned in the readme unfortunately didn't do the trick.

Thanks!

Trouble with "and" behavior queries

This isn't really an issue as much as me not understanding how "and" queries work. I'm trying to collect data for all Hispanic ex-pats from Mexico with the following query:

{   "name": "Hispanic ex-pats",
    "behavior": [
        {"and" : [6003133212372, 6003277229371], "name" : "Hispanic Mexican Ex-pats"}
    ],
     "geo_locations" : 
     [
        {
            "name" : "countries",
            "location_types" : ["home"],
            "values" : [
                "US"
            ]
        }
     ],
     "ages_ranges": [
        {"min":18, "max" : 25}
    ],
    "genders": [0]
}

But this generates an invalid query (according to Facebook). Do you know whether the query is written wrong?

(noob q): unable to install

Apologies if this is a very basic question, but I'm still learning my ropes with Python, and wanted to use this project. I carried out the installation instructions, and then initiated jupyter lab running a python 2 kernel to run the code. However, when I try to run watcherAPI, I keep running into the same error- "No Module named main". Any help would be greatly appreciated, I haven't been able to find any tutorial notebooks either, but I'm happy to write one once I'm able to do this.

screenshot

deprecated version of the Ads API?

�[32m2020-02-24 00:02:57�[0m �[35mMSI-GYOUZA�[0m �[34mroot[18388]�[0m �[30mERROR�[0m �[31mError Message:(#2635) You are calling a deprecated version of the Ads API. Please update to the latest version: v6.0.�[0m
�[32m2020-02-24 00:02:57�[0m �[35mMSI-GYOUZA�[0m �[34mroot[18388]�[0m �[30mERROR�[0m �[31m"{u'message': u'(#2635) You are calling a deprecated version of the Ads API. Please update to the latest version: v6.0.', u'code': 2635, u'type': u'OAuthException', u'fbtrace_id': u'ApMcc7aMH33Kblh9Q_oWDc4'}"�[0m

WARNING Facebook Error code 2

I can't run any example. I've got the "WARNING Facebook Error code 2" and the "WARNING Facebook Error Message: (#2) Service temporarily unavailable". In the "dataframe_skeleton_.csv" I obtained only NAs.

That happened to me if I either used my own token and application ID or one from your credential.

What should I do?

Thanks

Deprecated API

I've gotten the next error.

Error Message:(#2635) You are calling a deprecated version of the Ads API. Please update to the latest version: v14.0.

We should update API calls to a new version

Problem in Execution

Hello, For the below code, watcher.run_data_collection("pySocialWatcher/pysocialwatcher/quick_example.json")

Which file do you want to pass as quick_example.json? There is no such file in the folder. In the facebook credential I gave the user access token and respective user id for that app (see below), is it correct?
image

Please help.

How to handle "Sorry, something went wrong."

Hello, I noticed that the process fail when the Facebook API responds with a new error. It's not a json, but a html page.

The source of the web page is always the same:
<!DOCTYPE html>\n<html lang="en" id="facebook">\n <head>\n <title>Facebook | Error</title>\n <meta charset="utf-8">\n <meta http-equiv="cache-control" content="no-cache">\n <meta http-equiv="cache-control" content="no-store">\n <meta http-equiv="cache-control" content="max-age=0">\n <meta http-equiv="expires" content="-1">\n <meta http-equiv="pragma" content="no-cache">\n <meta name="robots" content="noindex,nofollow">\n <style>\n html, body {\n color: #141823;\n background-color: #e9eaed;\n font-family: Helvetica, Lucida Grande, Arial,\n Tahoma, Verdana, sans-serif;\n margin: 0;\n padding: 0;\n text-align: center;\n }\n\n #header {\n height: 30px;\n padding-bottom: 10px;\n padding-top: 10px;\n text-align: center;\n }\n\n #icon {\n width: 30px;\n }\n\n h1 {\n font-size: 18px;\n }\n\n p {\n font-size: 13px;\n }\n\n #footer {\n border-top: 1px solid #ddd;\n color: #9197a3;\n font-size: 12px;\n padding: 5px 8px 6px 0;\n }\n </style>\n </head>\n <body>\n <div id="header">\n <a href="//www.facebook.com/">\n <img id="icon" src="//static.facebook.com/images/logos/facebook_2x.png" />\n </a>\n </div>\n <div id="core">\n <h1 id="sorry">Sorry, something went wrong.</h1>\n <p id="promise">\n We\'re working on it and we\'ll get it fixed as soon as we can.\n </p>\n <p id="back-link">\n <a id="back" href="//www.facebook.com/">Go Back</a>\n </p>\n <div id="footer">\n Facebook\n <span id="copyright">\n &copy; 2018\n </span>\n <span id="help-link">\n &#183;\n <a id="help" href="//www.facebook.com/help/">Help Center</a>\n </span>\n </div>\n </div>\n <script>\n document.getElementById(\'back\').onclick = function() {\n if (history.length > 1) {\n history.back();\n return false;\n }\n };\n\n // Adjust the display based on the window size\n if (window.innerHeight < 80 || window.innerWidth < 80) {\n // Blank if window is too small\n document.body.style.display = \'none\';\n };\n if (window.innerWidth < 200 || window.innerHeight < 150) {\n document.getElementById(\'back-link\').style.display = \'none\';\n document.getElementById(\'help-link\').style.display = \'none\';\n };\n if (window.innerWidth < 200) {\n document.getElementById(\'sorry\').style.fontSize = \'16px\';\n };\n if (window.innerWidth < 150) {\n document.getElementById(\'promise\').style.display = \'none\';\n };\n if (window.innerHeight < 150) {\n document.getElementById(\'sorry\').style.margin = \'4px 0 0 0\';\n document.getElementById(\'sorry\').style.fontSize = \'14px\';\n document.getElementById(\'promise\').style.display = \'none\';\n };\n </script>\n </body>\n</html>\n

If I'm not wrong, at the moment the error is handle by the function handle_send_request_error() in utils.py. As this seems to be a temporary problem on Facebook, which can be the best way to handle it and trying to restore the process? By calling perform_collection_data_on_facebook()?

Thanks!

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.