CARTO Pass-Through API about soda-carto HOT 10 CLOSED

Now that I've actually read that doc page, it looks perfect!

from soda-carto.

from soda-carto.

This is almost working:

resources:
  Resources:
    ProxyResource:
      Type: AWS::ApiGateway::Resource
      Properties:
        ParentId:
          Fn::GetAtt:
            - ApiGatewayRestApi # our default Rest API logical ID
            - RootResourceId
        PathPart: carto # the endpoint in your API that is set as proxy
        RestApiId:
          Ref: ApiGatewayRestApi
    ProxyMethod:
      Type: AWS::ApiGateway::Method
      Properties:
        ResourceId:
          Ref: ProxyResource
        RestApiId:
          Ref: ApiGatewayRestApi
        AuthorizationType: NONE
        HttpMethod: GET # the method of your proxy. Is it GET or POST or ... ?
        MethodResponses:
          - StatusCode: 200
        Integration:
          IntegrationHttpMethod: GET
          Type: HTTP
          Uri: https://phl.carto.com/api/v2/sql # the URL you want to set a proxy to
          IntegrationResponses:
            - StatusCode: 200

The problem appears to be that API Gateway doesn't pass querystrings along by default, and it looks like the only way to enable them is to explicitly activate each one. Fortunately, with carto, we can activate q, format, and filename, though what if carto adds another parameter? And perhaps there are others that simply aren't in the sql api docs? Definitely not ideal :-/

Need someone familiar with API Gateway / cloudformation to weigh in. I've asked in freenode ##aws and the og-aws slack.

EDIT: Also, I think that ideally it would be data.phila.gov/api/v2/sql?q=xxxx for the URL schema, rather than /carto. Not super firm on it but it seems to make sense to me. Note that one of the socrata url structures is /api/views/.... but that shouldn't matter.

EDIT2: Apparently you can make the resource pathpart {proxy+} to use it like a "greedy parameter" and then use {proxy} in the method uri, like https://phl.carto.com/{proxy}. This may include querystrings. I haven't been able to get it to work. In particular, I was having trouble making the path something like /carto/{proxy+} or /api/v2/{proxy+}.

from soda-carto.

@andrewbt can you post the screenshots of the 3 configuration forms you got working when you get a chance? I can try to translate into cloudformation code this weekend.

from soda-carto.

Posted for help

from soda-carto.

Reposting screenshots from #37 :

Testable here: https://zsp6egcjhf.execute-api.us-east-1.amazonaws.com/dev/carto/v2/sql?q=select * from indego_stations where ST_DWITHIN(the_geom::geography, ST_GeographyFromText('POINT(-75.1636 39.9524)'),500)

Screenshots:

from soda-carto.

API Gateway allows you to “export” an API (under Stages) as a Swagger YAML or JSON document. One of the options for creating an API is through importing a Swagger document.

If we can’t get serverless.yaml to work with the HTTP-proxy integration and other options we need there for the pass-through API, Swagger seems like a good alternative and will still let us keep “deployment in code” rather than through the AWS console UI.

from soda-carto.

https://blog.jayway.com/2016/09/18/introduction-swagger-cloudformation-api-gateway/ 👯

from soda-carto.

We bloody cracked it!!

I got swagger to work, but it ended up overwriting all the serverless functions. But then I noticed that the swagger itself was basically cloudformation. So that told me what pieces were missing from the original cloudformation. And it's working! :D

https://67qq7fn418.execute-api.us-east-1.amazonaws.com/dev/carto/api/v2/sql?q=select%20*%20from%20pol_incidents_part1_part2%20limit%2010

from soda-carto.

from soda-carto.