220 lines
13 KiB
Markdown
220 lines
13 KiB
Markdown
+++
|
|
date = "2018-02-04T17:00:00-04:00"
|
|
title = "Citra Progress Report - 2017 Q4"
|
|
tags = [ "progress-report" ]
|
|
author = "anodium"
|
|
forum = 9777
|
|
+++
|
|
|
|
New year, new changes in Citra. Specifically, lots of technical changes under
|
|
the hood, from applets to IPC, have made Citra even more accurate and laid a
|
|
foundation for even more goodies the next year. With many great changes all
|
|
around, this article is going to be packed to the brim with all the new goodies
|
|
that have come to Citra during those crimson months.
|
|
|
|
But, enough faffing about! Let's get right into it:
|
|
|
|
## [citra-qt : Adding fullscreen mode](https://github.com/citra-emu/citra/pull/3001) by [Styleoshin](https://github.com/Styleoshin)
|
|
|
|
One of the most requested features for Citra has been enlarging the window to
|
|
cover the entirety of the screen. After almost a year of requests on both the
|
|
Citra Discourse and Discord servers, [Styleoshin](https://github.com/Styleoshin)
|
|
has finally delivered!
|
|
|
|
{{< figure src="/images/entry/citra-progress-report-2017-q4/fullscreen.png"
|
|
title="No more pesky window décor!" >}}
|
|
|
|
Simply go into `View → Fullscreen`, or just strike the Alt + Return keycombo, and
|
|
enjoy Citra to its fullest on your largest monitor.
|
|
|
|
## Rewrite AM Service and Add CIA Installation ([Here](https://github.com/citra-emu/citra/pull/2993), [here](https://github.com/citra-emu/citra/pull/3048), [here](https://github.com/citra-emu/citra/pull/2975), [there](https://github.com/citra-emu/citra/pull/3029), [this](https://github.com/citra-emu/citra/pull/3113), and [that](https://github.com/citra-emu/citra/pull/3144)) by [shinyquagsire23](https://github.com/shinyquagsire23) and [BreadFish64](https://github.com/BreadFish64)
|
|
|
|
The `am` service on a Nintendo 3DS handles the installation, tracking, and removal
|
|
of applications and games installed on the console and SD card(s) via a centralized
|
|
database in the NAND, with some parts of the database for SD card-installed titles
|
|
on the SD card itself. It has a handful of nice features such as installing packages
|
|
as they're streamed in, along with DLC support being cleanly implemented as a
|
|
side-effect of the CIA format's design. CIAs themselves are also designed to be
|
|
streamed in, which is used extensively across 3DS apps, such as the eShop streaming
|
|
downloads directly into `am` so that it installs at the same time, or `dlp` using
|
|
it to stream Download Play CIAs from one 3DS to another directly.
|
|
|
|
The CIA format starts off with a header that describes where each component of
|
|
itself starts within the stream and a bitfield. This is then followed by a
|
|
certificate signed by Nintendo, the CIA's ticket, and the title metadata (or TMD
|
|
for short). The TMD then contains a list of every CXI file the CIA could possibly
|
|
have, along with their name, size, etc.
|
|
|
|
{{< figure src="/images/entry/citra-progress-report-2017-q4/oot3dcia.png"
|
|
title="A diagram of *The Legend of Zelda: Ocarina of Time 3D*'s CIA" >}}
|
|
|
|
The astute reader might've noticed I said that the TMD contains a list of every
|
|
*possible* CXI. This is because a CIA might not contain every CXI for the game,
|
|
such as in the case of DLC. The bitfield I mentioned in the header is used to list
|
|
off which of the entries in the TMD actually exists within that CIA. This is
|
|
(ab)used in some cases, such as the Home Menu Themes, in that every single theme
|
|
that exists is actually a different CXI within the same CIA. Just tick the themes
|
|
the user owns in the bitfield and then attach those CXIs at the end of it. Dead
|
|
simple way for it to work perfectly with both streaming of the file, and making
|
|
personalized CIAs for each eShop user.
|
|
|
|
{{< figure src="/images/entry/citra-progress-report-2017-q4/menucia.png"
|
|
title="A tiny portion of the Home Menu Themes CIA" >}}
|
|
|
|
For the `am` side of streaming installations, the way it handles a request to
|
|
install an application is by creating and then giving the requester a handle to
|
|
a virtual file. This leaves most of the busy work to `am`, while the app only needs
|
|
to worry about writing the CIA file into that handle.
|
|
|
|
[shinyquagsire23](https://github.com/shinyquagsire23) reimplemented the entirety
|
|
of how Citra handles NCCH files, which quickly paved the way to implementing `am`
|
|
in Citra. This allowed users to use FBI to install applications from its virtual
|
|
SD card. He and [BreadFish64](https://github.com/BreadFish64) also went a step
|
|
further and added support for installing CIAs via the SDL and Qt frontends
|
|
directly, removing the need to use FBI. Do note that you cannot yet run installed
|
|
applications and games directly, but CXI executables can access the virtual SD
|
|
card for DLC, updates, etc.
|
|
|
|
## [Qtifw build installer](https://github.com/citra-emu/citra/pull/2966) by [j-selby](https://github.com/j-selby)
|
|
|
|
Citra now has a fancy new installer and updater, thanks to [j-selby](https://github.com/j-selby)
|
|
and [jroweboy](https://github.com/jroweboy)'s efforts! If you haven't already, we
|
|
strongly recommend you download and install Citra through it, since it will make
|
|
updating as easy as re-running the installer. You can check it out over [here](/download/),
|
|
|
|
## [macOS: Build x86_64h slice](https://github.com/citra-emu/citra/pull/2982) by [MerryMage](https://github.com/MerryMage)
|
|
|
|
A little known feature in macOS is fat binaries, the ability to have the same executable
|
|
contain binaries for multiple architectures. This was used extensively around 2007
|
|
to support Mac OS X's transition from PowerPC to x86_64, allowing developers to
|
|
have one binary work on all Macs effortlessly.
|
|
|
|
After poking around Apple's LLVM fork, [MerryMage](https://github.com/MerryMage)
|
|
found that they had a specific flag to build binaries for Intel Haswell and later.
|
|
Enabling it made a fat binary that contained both pre-Haswell and post-Haswell code,
|
|
allowing Citra to take advantage of the newer CPU's instructions, without dropping
|
|
support for older ones.
|
|
|
|
## [Kernel/IPC: Add a small delay after each SyncRequest to prevent thread starvation.](https://github.com/citra-emu/citra/pull/3091) by [Subv](https://github.com/Subv)
|
|
|
|
When communicating with services, titles on a real Nintendo 3DS expect to wait some
|
|
amount of time before the service replies or responds. Until now, Citra didn't implement
|
|
this, it responded without incrementing the virtual clock. From the emulated system's
|
|
perspective, it appeared as though services replied and responded literally instantly
|
|
because of this, leading to strange side-effects.
|
|
|
|
For example, some buildings in *Animal Crossing: New Leaf* fail to render and
|
|
cannot be interacted with.
|
|
|
|
{{< figure src="/images/entry/citra-progress-report-2017-q4/acnl-before.png"
|
|
title="Animal Crossing: New Leaf's acting up" >}}
|
|
|
|
Or worse, some games such as *Star Fox 3D* can't even
|
|
reach the title screen.
|
|
|
|
{{< figure src="/images/entry/citra-progress-report-2017-q4/fsUSER-threads.png"
|
|
title="A demonstration of how instant service replies can break a title" >}}
|
|
|
|
The example in this figure is taken directly from *Star Fox 3D*. When the game boots,
|
|
it checks if any save data exists on the console's SD card, and tries to recreate it
|
|
if it doesn't. It does this by sending a request to the service responsible for
|
|
file access, and then waiting for a reply. Because the request is synchronous,
|
|
the thread that made the request is put to sleep and its priority lowered. The
|
|
game actually uses this time that would otherwise be spent waiting by having other
|
|
threads run while the original thread was still asleep, so that by the time it
|
|
woke up with the response, it would have other resources it needed ready to go.
|
|
|
|
{{< figure src="/images/entry/citra-progress-report-2017-q4/acnl-after.png"
|
|
title="Animal Crossing: New Leaf, among other games, are now working perfectly with this change!" >}}
|
|
|
|
As Citra's responses are instantaneous from the point-of-view of the game,
|
|
the secondary thread doesn't have nearly enough time to finish its job. And, the
|
|
first thread's priority comes back up above the secondary's once it wakes up,
|
|
this leads to the first thread waiting on the secondary thread to finish, but it
|
|
never gets a chance to, due to its now much lower priority. Essentially, the
|
|
first thread waits on the secondary forever, because the secondary never gets a
|
|
chance to actually finish what it was doing.
|
|
|
|
[ds84182](https://github.com/ds84182) and [Subv](https://github.com/Subv) each
|
|
wrote homebrew software which found and measured the issue, respectively, which
|
|
[B3n30](https://github.com/B3n30) ran on a real Nintendo 3DS. With which, an
|
|
average delay for every type of service reply was found. Then, [Subv](https://github.com/Subv)
|
|
made Citra's virtual clock increment by this amount before fulfilling any service
|
|
request, solving many of the issues this brought.
|
|
|
|
## [core/arm: Improve timing accuracy before service calls in JIT](https://github.com/citra-emu/citra/pull/3184) by [MerryMage](https://github.com/MerryMage)
|
|
|
|
As our previous section demonstrates, Citra's virtual clock should be
|
|
incremented before any time code that was supposed to be on a real
|
|
Nintendo 3DS runs within Citra, including services. This change by [MerryMage](https://github.com/MerryMage)
|
|
makes services increment the clock *before* they are called, instead of after.
|
|
|
|
<!-- NOTE: Possibly hard to understand/too verbose? Reword if so. -->
|
|
One of the reasons this is important is that services can schedule calls to
|
|
other services to run in the future. If the current time in the virtual clock is
|
|
incorrect when the service schedule an event, it would run too early since
|
|
the time for the scheduled event would effectively be shifted back by the virtual
|
|
time it takes for the service that scheduled it would run.
|
|
|
|
Although this doesn't fix any reported bugs, it does make the timing emulation
|
|
of Citra much more accurate.
|
|
|
|
## [citra-qt : Fix a bug in our fullscreen implementation](https://github.com/citra-emu/citra/pull/3159) by [FearlessTobi](https://github.com/FearlessTobi)
|
|
|
|
In Citra's GUI, a strange bug exists, where if you have fullscreen enabled, and
|
|
you start a game while the window is maximized, the window instead unmaximizes
|
|
before starting the game. This was due to Citra not keeping track of the window's
|
|
state (be it normal, maximized, minimized, or fullscreen), and instead simply
|
|
defaulting to normal. [FearlessTobi](https://github.com/FearlessTobi) found,
|
|
[reported](https://github.com/citra-emu/citra/pull/3001#issuecomment-338401048),
|
|
and fixed the bug. Citra now keeps track of the window's state, position, and
|
|
size, every time it goes fullscreen and restores them every time it leaves
|
|
fullscreen.
|
|
|
|
## [Allow input configuration with SDL joysticks](https://github.com/citra-emu/citra/pull/3116) by [muemart](https://github.com/muemart)
|
|
|
|
In the input configuration menu, there has been the ability to change the key
|
|
bindings on the keyboard for years now. Unfortunately, this menu allowed rebinding
|
|
to keyboard keys only, not gamepads or any other input device one could use. This
|
|
led to guides such as [this one](https://community.citra-emu.org/t/temporary-controller-configurations-for-citra/1061)
|
|
being written, to help users manually change the configuration files if they
|
|
wanted to use gamepads with Citra.
|
|
|
|
This was extremely inconvenient and user-unfriendly to most, and so [muemart](https://github.com/muemart)
|
|
took it upon themselves to finally add support for configuring gamepads within the
|
|
configuration menu. Now, it's as simple as clicking a button, and pressing the
|
|
corresponding button on your controller, to set it up.
|
|
|
|
{{< figure src="/images/entry/citra-progress-report-2017-q4/controller.png"
|
|
title="Citra's input configuration menu" >}}
|
|
|
|
## [shader_jit_x64_compiler: Remove ABI overhead of LG2 and EX2](https://github.com/citra-emu/citra/pull/3145) by [MerryMage](https://github.com/MerryMage)
|
|
|
|
The shader JIT in Citra is a component of the video core responsible for
|
|
recompiling GPU shaders for the 3DS to x86 code, so that they can be run on the
|
|
user's CPU directly.
|
|
|
|
Shader instructions like LG2 (calculate binary logarithm) and EX2 (calculate binary
|
|
exponential) can be run potentially thousands of times per second in a typical
|
|
Nintendo 3DS title, so it's usually very worthwhile to try to optimize these as
|
|
best as possible.
|
|
|
|
In doing so, [MerryMage](https://github.com/MerryMage) actually rewrote these two
|
|
instructions to pure x86 assembly, and runs them inline with the rest of the instructions.
|
|
A faster algorithm, combined with not having to deal with the overhead of calling
|
|
an external library for math functions, led to this change almost halving the
|
|
amount of time it takes to calculate these!
|
|
|
|
{{< figure src="/images/entry/citra-progress-report-2017-q4/shaderjit.png"
|
|
title="Graph comparing times before and after this change" >}}
|
|
|
|
## Conclusion
|
|
|
|
As always, [changes big or small](https://github.com/citra-emu/citra/graphs/contributors?from=2017-10-01&to=2017-12-31&type=c)
|
|
are vital to the project. Brick by brick, Citra will eventually be an accurate enough
|
|
emulation of the Nintendo 3DS for most, if not all, uses. Thank you everyone for
|
|
the hard work you've poured in.
|
|
|
|
{{< figure src="/images/entry/citra-progress-report-2017-q4/promo.png"
|
|
title="And more to come." >}}
|