Stack alignment is unspecified about riscv-elf-psabi-doc HOT 39 CLOSED

@sorear Do you have time to look at this?

from riscv-elf-psabi-doc.

In the previous spec, we required 16-byte alignment for both RV32 and RV64 (mainly to support the Q extension).

As a special case, the RV32E ABI required only 4-byte alignment, since it is used in more memory-constrained environments and never has floating-point.

from riscv-elf-psabi-doc.

I came here from riscv/riscv-isa-manual#44.

The current RISC-V ISA 2.1 requires 16-byte alignment on hardware floating point implementations.
For soft-float it is not required. On the other hand C.ADDI16SP instruction assumes the stack pointer is kept 16-byte aligned and is useless if the stack pointer is not 16-byte aligned.

the RV32E ABI required only 4-byte alignment

Is this agreed officially? If it is, we need some notes on the description of C.ADDI16SP instruction.

from riscv-elf-psabi-doc.

The effect is is that C.ADDI16SP will be used less in RV32E. However, we expect most RV32E stack frames to be smaller, and so C.ADDI will suffice in many cases. In particular, C.ADDI16SP and C.ADDI together support stack frames with 4-byte alignment up to 32 bytes.

Can someone make a PR that documents 16B alignment? We can skip documenting the RVE 4B alignment until the RVE ABI is written down.

from riscv-elf-psabi-doc.

Thank you for your reply.

However, we expect most RV32E stack frames to be smaller, and so C.ADDI will suffice in many cases.

Do you have any background data?
I've been working embedded area. I do not have actual data which I can show you, but I have opposite feeling.
32byte (only 8 words) is not enough even for embedded applications.
Note that we cannot buy a small DRAM these years. But smaller code size is still important for power consumption and small cache memory.

On the other hand 16 registers are enough for many applications. RV32E with C extension fits embedded area very well.

I have been evaluating RISC-V architecture these months. One of issue for me is the lack of tool-chain support for RV32E. I hope the difference between non RV32E and RV32E is small to make too-chain support easier and earlier.

from riscv-elf-psabi-doc.

No, I do not. But most functions with large stack frames also have a lot of static code, in which case this issue is nearly irrelevant (2 uncompressible instructions => 4 extra bytes of code).

Also, if most functions have lots of stack-allocated variables, RV32E may not be the best choice--the extra registers in RV32I will reduce code size, and so may be worth the cost. The tradeoff point isn't clear, but if you think each regfile bit costs the same as 8-16 bits of code RAM, then something like 0.5-1 KB of extra code justifies RV32I over RV32E. For compiled C code, that probably means a total code size of 16-32 KB. Of course, it's highly application-dependent.

In any case, RV32E support in the toolchain is planned, but I don't have an ETA.

from riscv-elf-psabi-doc.

Do you intend 16-byte stack alignment only at function boundaries or everywhere?

Requiring greater than word alignment within functions (like ARM64) makes things more complicated for JITs like Dart and Cog, which have stack-based internal calling conventions and generally need to ensure stack slots have GC-safe values. It's possible to meet this requirement by using another register as the internal stack pointer and extra bookkeeping during transitions in and out of the runtime to make the ABI stack pointer safe, but I'd prefer to place the burden on signal handlers :)

from riscv-elf-psabi-doc.

Interesting question @rmacnak. Am I right in thinking that AArch64 is the only architecture that requires greater than word-size alignment within functions (rather than just at boundaries)? I'm possibly being dense, but it's not clear why they try to enforce this. Is it just collateral damage from the desire to have hardware-enforced checking of 16 byte alignment at function boundaries?

from riscv-elf-psabi-doc.

from riscv-elf-psabi-doc.

And the kernel really has to re-establish alignment, as it can't really rely on userspace software being well-behaved.

from riscv-elf-psabi-doc.

The only consequence is that badly behaved user programs would cause their own signal handlers to crash, so you can make an argument either way.

I'm not opposed to relaxing this restriction, it would have to happen very soon. @palmer-dabbelt what do you think

from riscv-elf-psabi-doc.

This (and the loosely connected issue of whether the kernel should restore tp/gp) are relevant for systems that use a non-C calling convention internally but need to deal with the possibility of C signal handlers, like Go.

from riscv-elf-psabi-doc.

@aswaterman: of course, good point.

I think 16-byte alignment on RV32 is unjustified, and it's probably not too disruptive to go ahead and loosen this requirement. It's difficult to judge if there's actually enough gain for the change to be worthwhile though. I don't think there's any ABI breakage from such a change (assuming systems using only standard extensions), but do speak up if I'm missing something. I could see the argument with sticking with 8-byte alignment across all RV32 variants (maybe with the exception of RV32E), for simplicity.

16-byte alignment on RV64 also seems weakly motivated. Q isn't part of the 'general' RV64G profile we expect to be most common and is almost definitely going to be used much, much less than SSE. Is it really too much to expect that functions using Q realign the stack? If you want to use Q registers beyond function boundaries you have to compile with a whole different ABI anyway.

from riscv-elf-psabi-doc.

Yeah, as I said in that email thread, I'd support changing the RV32 ABIs to use 8-byte stack alignment, if we don't think it will be disruptive. (RV32E should remain at 4 bytes.)

As for RV64, 16-byte stack alignment is nearly universal for 64-bit systems, not just x86.

from riscv-elf-psabi-doc.

(Also, changing the stack alignment to be less-than-2*XLEN-bit-aligned will break the calling convention, so I definitely do not support that change.)

from riscv-elf-psabi-doc.

I've thought more about this. It's very hard to imagine a realistic case where moving to 8-byte alignment on rv32 would break things, but it is technically an ABI change. If mixing code with 8-byte stack alignment with 16-byte stack alignment, stack objects with 16-byte alignment (e.g. int128_t or long double) may end up at 8-byte aligned locations. While no standard rv32 instructions require 16-byte aligned arguments, this could cause problems. Code could make assumptions based on these objects being properly aligned. It's hard to imagine wanting to calculate the memory address of the upper half of one of these objects and storing it in a register, however - previously ORI rd, rs1, 8 would have been semantically equivalent to ADDI rd, rs1, 8.

I'm not sure what the best route forward is, but the chance this change could cause issues isn't zero.

Changing stack alignment is obviously a bigger deal on x86 where you start getting exceptions for sse operations.

from riscv-elf-psabi-doc.

Indeed, it's not inconceivable. However, it is pretty unlikely (especially since GCC doesn't natively support 128-bit integers on 32-bit targets, AFAIK); and we aren't distributing compiled shared libraries of glibc yet, so I think we can get away with it.

from riscv-elf-psabi-doc.

FYI Linux already aligns the signal stack frame.

from riscv-elf-psabi-doc.

Andes would like to express our strong support for maintaining 16-byte stack alignment “all the time” and argue against relaxing the alignment to just function boundaries. The main reason is to reduce the interrupt latency which is very critical to real-time and embedded systems.

If we maintain constant stack alignment all the time, interrupt service routines do not need to perform any extra alignment. Otherwise, at the start of an interrupt, every interrupt service routine has to waste additional time to check and align the stack pointer before it can invoke any function calls. This is because an interrupt can occur on any instruction and not just on function boundaries. The interrupt service routine also has to restore the previous stack alignment before it is about to exit. The stack “adjusted value” needs to be saved properly (say, on the stack) if it allows preemption.

An alternative solution to reduce the interrupt latency is using HW to perform everything mentioned above and defining it as part of the privilege architecture. However, HW costs extra silicon area to implement and it still cannot completely reduce the alignment overhead to zero cycle. To complicate things further, HW has to keep the “auto-adjusted value” in some architecture state (such as Status Register) at the entry of an interrupt and base on it to restore the stack alignment at the exit of an interrupt. This architecture state must be carefully preserved and restored in the events of context-switching and nesting interrupts.

Therefore, based on feedback from numerous customers requesting for smaller interrupt latency, we like to urge you to reconsider restoring the original spec. of maintaining constant stack alignment all the time.

Thanks.

from riscv-elf-psabi-doc.

That's an interesting point. Most interrupt handlers begin by swapping sp with mscratch, so stack alignment is not a concern. But if the interrupt handler runs in the same stack, this is a problem.

This is a tradeoff, and it may be that we need to make different decisions for the Linux ABI and the embedded ABI. I'd like to hear other opinions on the matter; maybe you should email the sw-dev mailing list.

from riscv-elf-psabi-doc.

I agree this is something that might differ on Linux vs embedded. Stack alignment was mostly undocumented for quite some time in this repo, which became the primary point of reference for the RISC-V ABI and calling convention when that section was removed from the user spec. I hadn't realised it before, but you're right that previous wording was "the stack pointer is
always kept 16-byte aligned."

from riscv-elf-psabi-doc.

Another alternative is to require stack alignment at all times in the C ABIs, but allow the managed languages to make their own decisions about stack alignment (until they call into C ABI code, of course).

Either way, I think we'd satisfy both the embedded C programmers and the managed-language implementors.

from riscv-elf-psabi-doc.

Indeed, most non-OS systems only use one single stack for both foreground and interrupt handlers. For systems running RTOS, even though interrupt handlers does use a separate stack from normal tasks, the same problem will occur when there are nesting interrupts (because all interrupts use the same stack).

Thanks for your suggestion and I will initiate a discussion thread in the sw-dev forum.

from riscv-elf-psabi-doc.

@Kevin-Andes I've restored the old convention.

After thinking about it some more, it's not just an embedded issue. The same issue applies to operating systems' stack discipline when there's a nested interrupt. While we could have two ABIs, one for kernel and one for user, the benefit is so minor that it's not worth the effort.

Managed-language runtimes can still freely misalign the stack. This is just a C/C++/Fortran ABI issue.

from riscv-elf-psabi-doc.

I think this resolution keeps everyone happy. Are we good to close this issue?

from riscv-elf-psabi-doc.

seconded

from riscv-elf-psabi-doc.

 +aligned to 32 bits.  In the C, C++, and Fortran ABIs, the stack pointer must
 +remain aligned throughout procedure execution.  Other languages' ABIs are
 +free to define different stack-pointer alignment conventions but must realign
 +the stack pointer when invoking C, C++, or Fortran procedures.  The operating
 +system must realign the stack pointer prior to invoking a signal handler.

Should we limit C, C++, and Fortran for align everywhere only? so what about the other languages and assembly code? I think just left strict here and do not hold a white list.

from riscv-elf-psabi-doc.

Maybe "C-compatible ABIs" instead?

from riscv-elf-psabi-doc.

C-compatible ABIs sounds a little ambiguous? Java with JNI is C-compatible ABI or managed-languages?

from riscv-elf-psabi-doc.

It varies over time. When a JNI call happens, it must be C-compatible. During normal Java execution, it does not need to be C-compatible. Do you know better terminology to describe this concept?

from riscv-elf-psabi-doc.

Hummm, in fact, I prefer left strict without any exception, I mean just like user isa spec 2.1. if managed-language (and other languages) want to mis-align the stack that's their choice, keep the stack alignment consistent is simpler.

The RISC-V Instruction Set Manual Volume I: User-Level ISA spec 2.1, p.109

In the standard RISC-V calling convention, the stack grows downward and the stack pointer is
always kept 16-byte aligned.

from riscv-elf-psabi-doc.

Although managed language runtimes probably aren't following "the standard RISC-V calling convention", I think there's advantage in being clear to compiler about what they can/can't do. It's obvious that they can freely ignore the calling convention, but it's less clear which of the stack handling rules can be relaxed. This is really a system-level / runtime environment issue as much as it is an ABI issue.

I wouldn't want to see people waste time coming up with workarounds like https://community.arm.com/processors/b/blog/posts/using-the-stack-in-aarch64-implementing-push-and-pop which really aren't necessary on a standard RISC-V implementation running a Unix-style operating system.

from riscv-elf-psabi-doc.

As a matter of fact, Andrew’s latest update does NOT reduce the interrupt latency in many cases. For example, it is common for C/C++ environment to have a few lines of assembly (e.g., for optimization). Since assembly does not have the 16B alignment requirement (according to the new spec.), an interrupt that occurs while executing assembly will break the whole alignment assumption. As a result, all interrupt service routines still must perform the stack alignment because they cannot guarantee that the interrupt did not occur on the assembly portion. Therefore, the latest update did not reduce the interrupt latency which is critical for embedded and real-time systems.

In general, this problem would arise if a system has both “C-compatible” and “non-C-compatible” code. So restoring to the original spec. (always enforcing the alignment) seems to be an ideal solution.

from riscv-elf-psabi-doc.

@Kevin-Andes I don't think there is a problem. If your system requires the stack always be aligned, don't use code that's incompatible with the C ABI! Only use assembly code that is ABI-compliant, and only use compilers that generate assembly code that is ABI-compliant.

Mandating that the Oracle JVM adhere to the C ABI doesn't help this kind of system, because you would never execute the Oracle JVM on this kind of system.

from riscv-elf-psabi-doc.

I do think we need to fix the wording, though. I propose:

"In the standard ABI, the stack pointer must remain aligned throughout procedure execution. Non-standard ABI code must realign the stack pointer prior to invoking standard ABI procedures."

Then, if you avoid non-standard ABI code, everything is OK?

from riscv-elf-psabi-doc.

What you're effectively saying is that code which runs in the presence of non-realigning interrupt handlers has an additional constraint, to leave x2 aligned.

We are now up to 3 de facto ABI differences between microcontroller code and Linux user-space code:

1. User-mode time, cycle, instret CSRs trap instead of returning values when used from the hifive1 BSP
2. microcontroller environments frequently do not install the necessary unaligned access handlers, so microcontroller code must be compiled with -mstrict-align
3. microcontroller interrupt handlers do not realign sp, so assembly code intended for one cannot misalign sp between any pair of instructions (if interrupts are enabled)

from riscv-elf-psabi-doc.

Yeah.

from riscv-elf-psabi-doc.

In this case, we should clarify by adding some notice/reminder like:

“If a system contains non-standard ABI code, all interrupt service routines (ISR) must realign the stack pointer before invoking standard ABI procedures."

from riscv-elf-psabi-doc.

See #50

from riscv-elf-psabi-doc.