strikerx3 / strikebox Goto Github PK

Add XQEMU as a git submodule to this project and create an adapter for the following Xbox hardware components:

• NV2A
• APU
• ACI
• XID
• NVNet

The method currently in use by timing-sensitive threads such as the i8254 timer loop (which needs to trigger once every millisecond) is not accurate enough on Windows. In both computers I have access to, the loop does trigger 1000 times per second as expected, but it fires two events every ~2ms instead of one every 1ms.

Windows offers the following timer and timing-related APIs:

• QueryPerformanceCounter and QueryPerformanceFrequency: Useful for accurate time interval measurements, but not for triggering events as they require a busy loop that pegs down a CPU core to 100%.
• std::this_thread::sleep_until and std::chrono::high_resolution_clock: The technique currently in use. Cross-platform, but doesn't provide millisecond accuracy (on Windows at least).
• timeBeginPeriod and timeSetEvent: The most accurate of all methods I've tested, but only works if timeBeginPeriod(1) succeeds. (It should succeed on basically any modern computer using at least Windows 7 which is what StrikeBox targets.)
• CreateWaitableTimer and SetWaitableTimer: A synchronization object that fires periodically. Misses a lot of events.
• CreateTimerQueue and CreateTimerQueueTimer: Microsoft says this is a replacement for the deprecated timeSetEvent API. My tests argue otherwise.

Here's a snapshot of 100 milliseconds of execution using each technique. Time is measured in microseconds with QueryPerformanceCounter. Each dot represents an event fired by one of the methods at a given moment. The system was under light load at the time of the test, which should provide a realistic environment similar to the expected usage of the emulator. The code was compiled in 64-bit Release mode.

The graph describes exactly what the issue is with std::this_thread::sleep_until: it doesn't trigger frequently enough.

QPC triggers almost perfectly in sync with the expected tick rate, but it costs 100% of a CPU core and is still subject to thread preemption (as seen in the last tick).

CreateWaitableTimer/SetWaitableTimer and CreateTimerQueue/CreateTimerQueueTimer are unsuited to the task. They missed a lot of ticks and drifted away from the desired tick rate.

timeSetEvent is the best option out of all these. It's not perfect either, but we can't expect perfect accuracy from a non-realtime operating system. However, it is much better than the current technique.

I haven't tested this on Linux yet. In any case, if the std::this_thread::sleep_until method is not accurate enough, timer_create seems to be the solution.

Based on the detected MCPX and BIOS ROM versions (#11), we should be able to enforce specific hardware revisions, thus preventing illegal combinations of hardware and software from being configured by the user.

This could also be performed the other way around: based on a hardware revision to be emulated, list all valid MCPX/BIOS ROMs (from a set of files provided by the user) that can be selected with it.

Did you and @mborgerson discuss this?

Find out if there is a reliable method to identify if the BIOS ROM provided is a retail or debug version, what version of the kernel it implements, and perhaps go even further and detect if it is a hacked BIOS.

The same applies to the MCPX ROM. This should be much easier to detect.

One possible method is to maintain a database of known ROM dumps along with their hashes. The database will provide all the metadata necessary to identify the ROM.

Integrate XQEMU's NVNet emulation on the XQEMU adapter (#22).

HAXM also supports Mac OS X. We need to modify the existing HAXM module to support that operating system.

XQEMU's USB emulation and XID are necessary for input emulation.

Also check out QEMU's USB emulation.

Another possibly useful source is VirtualBox.

There was also ergo720's OHCI-LLE branch (user deleted) which has been merged with Cxbx-Reloaded's develop branch a while ago. It doesn't work yet, but may serve as a starting point.

Of course, it's also possible to go straight to the source and read the specifications: here or here.

Mac OS provides the Hypervisor.Framework as it's virtualization platform.

We want to implement this as a Mac-only CPU module.

Some games depend on the TSC counting at exactly 733333333 counts per second, otherwise their physics break, they run too fast, or other issues may arise.

The goal here is to research options to handle the RDTSC instruction, including features provided by the hypervisors and hardware, and implement them.

The current image loaders are very basic. They rely on the C file management functions (fopen, fread, ftell, etc.), have no caching mechanism, don't implement write-on-copy as advertised, have poor performance, and so on. They also only support raw hard disk images and game discs in the XISO format.

The current implementations live in core/strikebox/hw/ata/drvs/drv_vhd_image.(cpp|h) and core/strikebox/hw/ata/drvs/drv_vdvd_image.(cpp|h).

For this issue, the goal is to:

• Create an abstraction for image loaders in order to support different formats, such as QEMU's QCOW2 format and other popular formats, perhaps even a custom format that embeds media information if that turns out to be necessary, and virtual disks/DVDs that directly access host folders;
• Improve performance by using platform-specific features such as memory mapped files; and
• Pave the way for new features such as the ability to read audio CDs or video DVDs in the emulator.

It may be necessary to implement new ATA/ATAPI/SCSI commands to support these features. Refer to the following specifications and manuals:

• AT Attachment with Packet Interface Extension (ATA/ATAPI-4)
• SCSI Architecture Model - 3 (SAM-3)
• SCSI Primary Commands - 3 (SPC-3)
• SCSI Block Commands - 3 (SBC-3)
• SCSI Multimedia Commands - 3 (MMC-3)

• PIC
• PIT (System timer)
• Super I/O (Debug Kits and Dev Kits only)
  • Serial ports
  • Character drivers
    • Null driver
    • Windows serial ports
    • Linux serial ports
    • stdin/stdout (low priority)
    • Possibly others, such as TCP, UDP, file and more (low priority)
• CMOS (MC146818 RTC) (#14)
• SMBus
  • EEPROM: a simple .bin file should do (#7)
  • SMC
    • AV pack
    • Conexant TV encoder
    • Power/reset controls
    • LED
    • Fan controller
    • DVD tray
    • Temperature monitor (ADM1032) (#6)
• PCI bus
  • LPC / ISA bridge
    • MCPX ROM overlay (#13)
    • ACPI timer
    • TV encoder
    • Possibly others
  • PCI bridge
  • AGP bridge
  • IDE controller (#10)
    • ATA/ATAPI-4 / SCSI (#10)
  • NV2A (#4)
  • USB (OHCI) (#9)
    • XID devices (#9)
  • MCPX APU (#5)
    • Front-end Engine (FE)
    • Voice Processor (VP)
    • Global Processor (GP) (compatible with the Motorola DSP56300)
    • Encode Processor (EP)
  • MCPX ACI / AC97 (#5)
  • NVNet (#8)

Device tree based on this, this and other sources.

Windows Hypervisor Platform is a virtualization platform based on Microsoft's Hyper-V released with the April 2018 Update for Windows 10. It must be enabled manually from the Windows Features panel.

We want to implement this as a Windows-only CPU module.

Integrate XQEMU's NV2A emulation on the XQEMU adapter (#22).

This can be based on XQEMU's implementation of the hardware.

The LPC bus code is emulating the basics of several components, including the MCPX ROM overlay. The ROM itself is mapped as a GPA range at the top of the x86 address space from 0xFF000000 to 0xFFFFFFFF and is manually filled up with bytes from the BIOS ROM. The last 512 bytes are replaced with the MCPX ROM if that is enabled. It is then restored once the disable bit is set.

The correct way to emulate this would be to attach devices to the LPC bus, one of them being the MCPX, and let MMIO flow through to the device in order to access the BIOS or MCPX ROM data depending on the state of the overlay. The GPA range reserved for the ROM would no longer be necessary.

In order to persist the EEPROM's contents, we need to save it to a file. There should be a setting that allows the user to specify which file they want to use. Create a sensible default EEPROM with generated serial number and default parameters in case it does not exist. Use memory mapped files if available on the platform, so that changes to the EEPROM are automatically persisted to disk.

The project is missing a LICENSE. I'm thinking of something permissive, such as MIT or Apache 2.0.

All files must have their headers updated to include the license text.

The previous name was more of a placeholder and it has come to my attention that it has unfortunate connotations too, which would make it hard to search for. The current name is also a placeholder, but certainly an improvement and a candidate for the final name.

The final name can be discussed further in this issue, but for now we don't need anything clever or creative, just a better placeholder.

With most of the IDE emulation implemented, we're now booting into the Dashboard. The system does not load game discs yet because it thinks that the clock settings are incorrect, due to the lack of CMOS emulation, and force boots into the dashboard to fix the settings.

XQEMU uses the MC146818 RTC emulator with some modifications (header, registers, source). Here is the chip's datasheet.

virt86 provides an abstraction of x86 virtualization platforms originally based on StrikeBox's CPU abstraction framework. It has improved and fixed many bugs (for example, WHPX now initializes correctly) and is likely to be improved by the community.

This includes the hard drive and the DVD-ROM. Ideally, we want to be able to emulate the DVD-ROM in such a way that allows us to read from XISO files or extracted folders. If possible, do the same for the hard drive.

Links to IDE emulation code: XQEMU, QEMU, VirtualBox.
Matt Borgerson made an userspace FATX driver here which can be useful for understanding the hard drive file system.

The game discs use the XDVDFS file system and there are tools that can read them. However, they only deal with the data partition; the disc actually contains hidden partitions that hold authentication information and describe the media's contents. We need to keep this in mind if we're to emulate a DVD drive at a low level. (Or use a modified kernel that skips media checks.)

The Xbox uses an ATA/ATAPI-4 controller. The official specifications can be found here. There's also a PCI Bus Master IDE controller that uses DMA for high speed transfers. Some references for this device: here and here.

Additionally, for accessing the DVD drive, the SCSI command protocol is used under the PACKET ATA command. The specification for SCSI commands can be found here and an overview of operation under the ATAPI protocol is described here.

Integrate XQEMU's APU and ACI emulation on the XQEMU adapter (#22).