Handle types related to variable arguments better in the Checked C rewriter about checkedc-clang HOT 4 CLOSED

I looked at the handling of variable argument in clang more closely while fixing a bug. Here is a summary:

• clang does not seem to know about the va_list type itself. According to the C spec, the va_list type must be a complete object types.
• clang creates a special built-in type __builtin_va_list. Clang appears to recognize that the type needs to be around when va_list is around. The mechanism for triggering this was unclear. The use of two underscores in the beginning of the name means that this is an identifier reserved for system use.
• The way that variable argument information is represented is platform-specific and considered part of the platform ABI. Clang typedef's __builtin_va_list to the platform-specific type. The function ASTContext::getBuiltinVaListDecl returns the typedef declaration.
• The built-in type is used to type variable argument-related primitive operations.

It seems that the rewriter will need to specially recognize the type named va_list. The rewriter should specially recognize the name typedef name __builtin_va_list. There are a wide variety of platform-specific types used to represent variable argument information, including char * and void *. It is not practical to avoid rewriting those types because they could be used in ways completely unrelated to variable arguments.

The rewriter also needs to handle source-level typedefs that map to va_list. We should have a function that recognizes type names that should not be rewritten because they are special names related to variable arguments.

There are two places in the rewriter code that try to handle va_list-typed variables: specialCaseVarIntros in ConstraintBuilder.cpp and PointerVariableConstraint in ProgramInfo.cpp. The former seems to only be applied to local variable definitions and looks for types named "va_list". It misses parameters. The latter checks all pointer types, but it only looks for the x86-x64 ABI-specific type to which __builtin_va_list is typedef'ed. A bug fix added recognition of va_list. It is unclear why the rewriter needed to recognize the the type to which __builtin_va_list is typedef'ed for x64. This suggests that either clang is using the desugared name when typing built-in primitives or that the rewriter is desugaring a name and losing typedef information. It is also unclear whether all the recognition can be done in PointerVariableConstraint and specialCaseVarIntros can be drooped.

from checkedc-clang.

I think the thing that was hanging me up was that there are three things you can do with a Type: you can look at it in its current form, you can de-sugar it one "step", or you can take a big step to de-sugar it fully. I think a principled thing to do would be to take a given type and either de-sugar it one step or traverse it with the appropriate Visitor looking for va_list. At the time, I didn't have the bandwidth to do that. Maybe now is the time?

from checkedc-clang.

I think you can put this off for now. At this point, I think it is better to focus on enhancing and evaluating the rewriter.

from checkedc-clang.

Closing this work item as stale. The convert tool has been mostly rewritten (see PR #642). We can reopen this or create a new issue if we decide further work is needed.

from checkedc-clang.