Filter grabs incorrect data (Similar to Issue #6) about tableau-scraping HOT 4 CLOSED

Works perfectly now, thank you so much!

With regard to the sessions issue: I'm guessing that's something specifically related to how this agency set up their server. I hadn't thought about this until just now, but this chart and others hosted on the same server will often reset themselves when switching between filters in a browser.

Thank you again for coming up with a solution for both issues and for doing it so quickly.

from tableau-scraping.

@jbadelson Hello, the website seems to be only available in the US. I've managed to run it in repl.it (based in US) and extract all metadata for analysis

After investigating, the problem comes from the filter index, but my guess is that table.schema[].ordinal field seems to indicate the start filter index:

"schema": [{
    "caption": "Region",
    "collation": {
        "f": 0,
        "l": 4294967295
    },
    "column_class": 1,
    "dataType": "s",
    "family_name": "Extract",
    "fieldType": "N",
    "hidden": false,
    "name": ["sqlproxy.10q6ok10kzhj6r13vx7j91uvn4mv", "Region"],
    "ordinal": 1, <==================================================
    "role": "d"
}],

I didn't have any example showing that this field was important until now. The 3 following setFilter usecases have all ordinal set to 0:

https://replit.com/@bertrandmartel/TableauCovidNewHampshire
https://replit.com/@bertrandmartel/TableauCovidSouthCarolina
https://replit.com/@bertrandmartel/TableauFilter

I will implement the parsing of the ordinal field in schema object

I think it's more related to #5 because it deals with how filterJson parsing is implemented. Indexing issues are major challenges in this library since the indexing is sparsed in different places. For example, filter and select have completly different indexing mecanism:

• select:
  • selection data is located in the data json data
  • data is located in the same location as the worksheet data except that each selectable column has isAutoSelect field set to true
  • data is located in dataDictionary field and indexing info in the paneColumnsData field
• filter:
  • filter data is located in the info json data
  • data is located in a json called filtersJson and which has a tuples field mapping keys/values

from tableau-scraping.

@jbadelson I've released v0.1.9 which fix the index issue. But there is still something amiss when working with this url. I don't know if it's related to the filters, but it seems it doesn't return data (KeyError: 'dataDictionary') if I perform more than X setFilter in a row. For instance:

from tableauscraper import TableauScraper as TS

url = 'https://analytics.la.gov/t/LDH/views/covid19_hosp_vent_reg/Hosp_vent_c'
ts = TS()
ts.loads(url)
workbook = ts.getWorkbook()

ws = ts.getWorksheet('Hospitalization and Ventilator Usage')

regions = next(iter([
    t["values"]
    for t in ws.getFilters()
    if t["column"] == "Region"
]))
print(regions)

for region in regions:
  print(region)
  wb = ws.setFilter('Region', region)
  ws = wb.getWorksheet('Hospitalization and Ventilator Usage')
  print(ws.data)

It fails at 5 - Southwest (or sometimes 8 - Monroe) but if I restart the session (with a new TS object) I get the result for 5 - Southwest so the filter query seems correct.
Something is wrong with the session and I think it's related to how the dataDictionary is managed between calls (maybe tableau server specific). I still need to investigate but you can workaround by re-instantiating the TS object for the moment if you need to iterate all the regions

This seems particularly unusual since https://replit.com/@bertrandmartel/TableauCovidNewHampshire#main.py for NewHampshire works well with more than 10 setFilter in a row

from tableau-scraping.

@jbadelson Everything is fixed in v0.1.10. The following get all regions:

from tableauscraper import TableauScraper as TS

url = 'https://analytics.la.gov/t/LDH/views/covid19_hosp_vent_reg/Hosp_vent_c'
ts = TS()
ts.loads(url)
workbook = ts.getWorkbook()

ws = ts.getWorksheet('Hospitalization and Ventilator Usage')

regions = next(iter([
    t["values"]
    for t in ws.getFilters()
    if t["column"] == "Region"
]))
print(regions)

for region in regions:
  print(region)
  wb = ws.setFilter('Region', region)
  regionWs = wb.getWorksheet('Hospitalization and Ventilator Usage')
  print(regionWs.data)

repl.it: https://replit.com/@bertrandmartel/TableauCovid19Louisiana

Note that I've added a warning in case the response doesn't return any data. In this case, it's normal behaviour but most of the time, it would mean that something is wrong in the input or that there is a bug

from tableau-scraping.