azure / azure-functions-language-worker-protobuf Goto Github PK

We currently have a hard copied version of the official protobuf duration.proto file so that we can reference it in our custom FunctionRpc.proto file. This causes errors with the java language worker, as it tries to compile duration.proto, which is already defined.

This file was included because it is necessary for compilation in the nodejs and python language workers. However, this shouldn't be necessary. For the python and nodejs, we should see if the language-specific libraries we are using for compilation have updated ways of handling this dependency.

Since generated files will come from other repos, and other repos already .gitignore any change to this repo, having a .gitignore is a noop and is confusing.

Context: Azure/azure-functions-nodejs-worker#59

From @ahmelsayed on April 30, 2018 21:7

From @graham-sportsmgmt on February 17, 2018 3:49

[email protected]
JavaScript HttpTrigger
Local function emulator on Windows 10 x64

I am trying to set a response header in order to flag the length of the result set to our front end code.

This method throws an error:

context.res = {
  "headers" : { "length": response.length },
  "body": response
};

invocationResponse.outputData.data.http.headers: string{k:string} expected

This method doesn't throw an error, but adds no header to the response:

context.res.setHeader('length', response.length);

Copied from original issue: Azure/azure-functions-core-tools#386

Copied from original issue: Azure/azure-functions-host#2768

There's a bug with git where merge subtree causes a fatal: refusing to merge unrelated histories error.

Change update instruction (2.) to be:
git merge -s subtree proto-file/dev --squash --allow-unrelated-histories

with backup command: git merge -X subtree=<path in language worker repo> --squash proto-file/<version branch> --allow-unrelated-histories

cc @alrod

Right now, when receiving an invocation request, you need to check the binding type and construct the typed binding yourself...

(See here and here)

The point of having a protobuf definition should include having the typed bindings.

As part of the next breaking change release, we need to make sure we go back over the protobuf and remove all unused properties and messages such as:

• WorkerHeartbeat
• worker_version
• Maybe RpcHttp
• managed_dependency_enabled - #34
• Updating the case of the "Properties" field to snake_case for consistency (as per the style guide we follow)

Issue

If a worker function wants to, say, write to a table output binding, and there is a problem with that binding, there is no way to inform the worker function that the table binding is incorrect during execution so that it can take appropriate action, therefore CreateIfNotExists and other patterns are not possible without leveraging direct SDKs, which kind of defeats the point of using outputbinding in the first place.

Recommended Resolution

Allow workers to submit Output Bindings ahead of final InvocationResponse, and the host can return an immediate result to the worker so that the worker can inform the function if required so it can take appropriate action if there is a problem.

This would be an RPC function definition like SubmitOutputBinding or something.

References

Azure/azure-functions-powershell-worker#284

We need to add tests to verify proto file updates that generate code for all the languages we support.

We are creating a new message for the Host to tell the worker to Initialize the function app.
For now, it will include two properties: function app root, and managed dependency enable option.

As a result, we will no longer need the managed_dependency_enabled property in FunctionLoadRequest--we need to remove this property once all workers start using this new message.

We need to allow users to set multiple cookies on the HttpResponse, since we can't fold multiple cookie values into one header and the HttpRequest object does not support duplicated headers.

When you add multiple Set-Cookie headers to the response, actually they are combined and only one header is sent with comma-separated cookies. According to RFC2109 it is a valid syntax. But it is not according to RFC6265, which deprecates RFC2109

Proposal:

// Http cookie type. Note that only name and value are used for Http requests
message RpcHttpCookie {
    // Enum that lets servers require that a cookie shouoldn't be sent with cross-site requests
    enum SameSite {
        Undefined = 0;
        Strict = 1;
        Lax = 2;
    }
    
    // Cookie name
    name: string,
    
    // Cookie value
    value: string,

    // Specifies allowed hosts to receive the cookie
    domain: NullableString,

    // Specifies URL path that must exist in the requested URL
    path: NullableString,

    // Sets the cookie to expire at a specific date instead of when the client closes
    expires: NullableString // becomes Nullable<DateTimeOffset> in C#, maybe use google.protobuf.Timestamp?

    // Sets the cookie to only be sent with an encrypted request
    secure: bool, 

    // Sets the cookie to be inaccessible to JavaScript's Document.cookie API
    http_only: bool,

    // Can restrict the cookie to not be sent with cross-site requests
    same_site: SameSite,

    // Sets the cookie to expire after a specific length of time instead of when the client closes
    max_age: NullableString // becomes Nullable<TimeSpan> in C#, maybe use google.protobuf.Duration?
}

git fetch proto-file refs/tags/v1.1.0-protofile

git merge -X subtree=src/WebJobs.Script.Grpc/azure-functions-language-worker-protobuf --squash v1.1.0-protofile --allow-unrelated-histories

Causes buffer overrun, first reported here: Azure/azure-functions-nodejs-worker#142

The root cause of this issue is that we use a "map" in our protobuf file.

From the protocol buffers docs:

If you provide a key but no value for a map field, the behavior when the field is serialized is language-dependent. In C++, Java, and Python the default value for the type is serialized, while in other languages nothing is serialized.

In combination with protobuf.js, this results in generated code that overflows.

                    case 15:
                        reader.skip().pos++;
                        if (message.query === $util.emptyObject)
                            message.query = {};
                        key = reader.string();
                        reader.pos++;
                        message.query[key] = reader.string();
                        break;

There are important files that Microsoft projects should all have that are not present in this repository. A pull request has been opened to add the missing file(s). When the pr is merged this issue will be closed automatically.

Microsoft teams can learn more about this effort and share feedback within the open source guidance available internally.

Merge this pull request

Need to make this change to flow headers and query even if the values are empty/null
Root cause: #21