Citra Progress Report 2017 August (#48)

* Added the first draft of the July 2017 Progress Report

whoop

* Removed unecessary files

* First round of changes suggested

Every comment I said fixed on has been addressed. Most reviews having to do with the Markdown/HTML formatting of the article have not been addressed yet, though.

* Quickfix EVERYTHING

* LET'S FIX THINGS FASTER YAAAAY

* Add coauthor

* Quickfix: Review by j-selby

We really should've dropped this report a long time ago. =_=

* Refactor 2017july into 2017august

* Refactor to 80 cols

* Rewrite intro and outtro

* Add candidate PRs

* Remove #2861

* Write #2766 #2822 #2871 #2872 #2891

* Trim headers

* Organize PRs

* Write #2850

* Write #2869

* Fix nit @Subv

* Refactor #2869 to 80 cols

* Add {{% title %}} shortcode

* Change <img /> to {{< figure >}}

* Apply fixes suggested by @j-selby

* Reword #2869

* Apply fixes suggested by @bunnei

* Apply fixes suggested by @jroweboy

* Apply fixes suggested by @j-selby

* Apply fixes suggested by @lioncash

* Apply nit by @j-selby

* Apply fixes suggested by @Subv

* Remake theme-comparison.png

* Add 2017august banner by @bunnei

* Retake Screenshots

Monitor Resolution: 4096x2160
Internal Resolution: 4000x2400 (10x)
External Resolution: 3467x2080

* Fix metadata

site/content/entry/citra-progress-report-2017-august.md

@@ -0,0 +1,172 @@
++++
+date = "2017-09-16T00:28:00-04:00"
+title = "Citra Progress Report - 2017 August"
+tags = [ "progress-report" ]
+author = "anodium"
+coauthor = "saphiresurf"
+forum = 3566
++++
+
+2017 has been an amazing year, with more work having been put into the project
+than ever before, but it's not over yet! Last we met was
+[June](/entry/citra-progress-report-2017-june/), and just two months later the
+[Citra issue tracker](https://github.com/citra-emu/citra/pulls) is brimming with
+lots of changes once more. I am extremely excited for this month (and what's coming
+up the next few months) but we're getting ahead of ourselves! On this progress
+report, let's check out the big fish in the July and August pond of patches!
+
+## Updating The Software Renderer ([This](https://github.com/citra-emu/citra/pull/2766), [that](https://github.com/citra-emu/citra/pull/2822), [here](https://github.com/citra-emu/citra/pull/2871), [there](https://github.com/citra-emu/citra/pull/2872), and [those](https://github.com/citra-emu/citra/pull/2891)) by [wwylele](https://github.com/wwylele) and [Subv](https://github.com/Subv)
+
+Citra has two main rendering backends, software and OpenGL, but until very
+recently, no one gave much attention to the software backend. The OpenGL backend
+is faster, makes better use of the GPU, and allows things such as texture forwarding
+for higher resolution rendering. But there is one thing on which the OpenGL backend
+falls flat on its face---accuracy.
+
+However, [wwylele](https://github.com/wwylele) has just revived the software
+renderer (inspired by an attempt [Subv](https://github.com/Subv) made prior),
+adding almost all of the features the hardware renderer had received over the years.
+In fact, every addition to the hardware renderer has been given an equivalent
+in software:
+
+{{% table %}}
+|      Feature      |                        Hardware                       |                                                   Software                                                   |
+| ----------------- | ----------------------------------------------------- | ------------------------------------------------------------------------------------------------------------ |
+| Fragment Lighting | [#1264](https://github.com/citra-emu/citra/pull/1264) |  [#2766](https://github.com/citra-emu/citra/pull/2766), [#2822](https://github.com/citra-emu/citra/pull/2822) |
+|   Spot Lighting   | [#2727](https://github.com/citra-emu/citra/pull/2727) |                             [#2871](https://github.com/citra-emu/citra/pull/2871)                            |
+|  Geometric Factor | [#2776](https://github.com/citra-emu/citra/pull/2776) |                             [#2872](https://github.com/citra-emu/citra/pull/2872)                            |
+|    Bump Mapping   | [#2762](https://github.com/citra-emu/citra/pull/2762) |                             [#2891](https://github.com/citra-emu/citra/pull/2891)                            |
+{{% /table %}}
+
+Despite the software backend being incredibly slow, it is important to have a
+complete software implementation of the 3DS' GPU so that Citra can be used as a
+stable, working, and complete reference implementation in the future, when
+obtaining a working real console may be much more difficult.
+
+## [Handle Invalid Filenames When Renaming](https://github.com/citra-emu/citra/pull/2850) by [j-selby](https://github.com/j-selby)
+
+Citra emulates the 3DS system services at a high level of emulation, or HLE for
+short. What this means is that every time a 3DS application or game running in
+Citra makes a request to the 3DS System Software, Citra captures the request and
+tries to translate it into its PC equivalent, rather then running the 3DS System
+Software directly.
+
+For example, if a game makes a call to [`FS:OpenFile`](https://www.3dbrew.org/wiki/FS:OpenFile),
+Citra in turn calls the operating system's file opening function
+([`_wfopen_s()`](https://docs.microsoft.com/en-us/cpp/c-runtime-library/reference/fopen-s-wfopen-s) on Windows,
+or [`open()`](http://pubs.opengroup.org/onlinepubs/9699919799/functions/open.html) on macOS and Linux),
+with [the path to the virtual SD card](/wiki/user-directory/) added to the beginning.
+
+Now, on top of the usual names for files and folders, there's two special folders
+inside every single folder on your computer, `.` and `..`. These aren't actual
+folders in the sense that you can place files and other folders inside of them.
+Instead, they each symbolize the current folder, and the folder one level above
+it, respectively. For example, `C:/Windows/System32/..` actually means `C:/Windows/`.
+
+With this in mind, a few Citra developers believed a game could, in theory, chain
+multiple `..`s together to get to a file they weren't supposed to know even existed,
+like `/../../../../Documents/IMPORTANT.docx`. Citra would then ask the operating
+system to open the file `%AppData%/Citra/sdmc/../../../../Documents/IMPORTANT.docx`,
+which *actually* means that it would open `C:/Users/Anodium/Documents/IMPORTANT.docx`!
+
+Before you suggest that Citra simply ignore `..`, a game can use it for legitimate
+purposes. And if it were to use it, it would most likely crash, as the resulting
+file path wouldn't exist.
+
+This was already handled for most file functions in Citra, opening, reading,
+writing, etc. except for a few things, like renaming a file. In this case, the
+malicious game could just ask Citra to change `/../../../../Documents/IMPORTANT.docx`'s
+name to `/delicious_secrets.docx`, effectively moving `IMPORTANT.docx` into Citra's
+virtual SD card! From there, the game could just ask Citra to open
+`/delicious_secrets.docx` to read the file it was originally forbidden from accessing.
+
+This patch now fixes this, such that if any 3DS game or application tries to do
+exploit the rename file function from inside Citra, rather than doing what it asks,
+Citra gives them `ERROR_INVALID_PATH`, which most games interpret by crashing. So
+far, we haven't found any real 3DS software that tries to do this, but at least
+now future attempts to do so are blocked.
+
+Do note though, that most of us are not security experts, and even for those who are,
+harderning the software is much more difficult and much less worthwhile than actually
+having an accurate 3DS emulation. The surface area of an emulation project is huge,
+and so writing malicious code intending to exploit one is not too difficult in any case.
+
+But at the same time, malware authors generally target either the largest populations,
+or the most lucrative populations, and emulation is neither. And for the effort
+required to do so, most will only toy with the idea in their head, or maybe write
+a proof-of-concept for fun, rather than genuine malicious intent.
+
+## [Fix Edge Cases for TextureCopy](https://github.com/citra-emu/citra/pull/2809) by [wwylele](https://github.com/wwylele)
+
+The 3DS GPU has a data transfer mode called `TextureCopy`, which as the name says,
+is for copying textures but with a configurable gap in case the texture is going
+to be copied into a smaller resolution area. This mode is enabled by setting the
+3rd flag in the GPU and causes it to ignore every other flag except the 2nd, which
+is used to tell it whether or not to crop the texture. 
+
+This is nice because it can be used as a quick and easy way for 3DS developers to
+duplicate textures, and can be used in situations such as the one pictured running
+on hardware in Pokémon Super Mystery Dungeon below. But when it came to running
+something that took advantage of this feature in Citra, it didn't always work the
+same as it would on console.
+
+{{< figure src="/images/entry/citra-progress-report-2017-august/texturecopy-before.png" 
+    title="How jagged" alt="Pokémon Super Mystery Dungeon During Deoxy's and Rayquaza's Face Off   I N   S P A C E" >}}
+
+Fortunately (and to much rejoicing!) wwylele stepped into the ring to wrestle with
+this issue. They prepared a [test program](https://github.com/wwylele/ctrhwtest/tree/master/texture-copy-test)
+to help gain an understanding of how the hardware handles the TextureCopy operation
+in comparison to Citra. After the hard work of doing the research was out of the
+way, wwylele implemented it in Citra.
+
+{{< figure src="/images/entry/citra-progress-report-2017-august/texturecopy-after.png" 
+    title="Deoxys is having a bit of a hard time, no?" alt="Pokémon Super Mystery Dungeon During Deoxy's and Rayquaza's Face Off   I N   S P A C E" >}}
+
+## [Use Docker For Linux Builds](https://github.com/citra-emu/citra/pull/2869) by [j-selby](https://github.com/j-selby)
+
+Ubuntu Linux 14.04 is the de-facto standard desktop Linux distribution. It's also
+old. [*Very* old](https://wiki.ubuntu.com/TrustyTahr/ReleaseSchedule#line-37).
+So old, in fact, that the compiler it ships with can't compile Citra. And our
+buildbot, [Travis CI](https://travis-ci.org/), that automatically compiles and
+builds Citra from source, just so happens to use Ubuntu 14.04 VMs. *`:(`*
+
+Formerly, we would update the compiler from a third-party repository before compiling
+Citra itself. This also had the side-effect of updating the standard library that
+comes with the compiler, as each compiler version is inextricably tied to the same
+version of library by design. Unfortunately, a recent update to the library was
+incompatible with a large majority of systems because it's too new for Ubuntu 14.04,
+breaking the Linux build once again.
+
+Now, rather than building Citra directly inside Travis, a
+[Docker container](https://www.docker.com/) is started that's running Ubuntu 16.04
+instead, which is much more well supported (and yes, it can compile Citra out of the box!).
+
+## [UI Themes](https://github.com/citra-emu/citra/pull/2804) by [Kloen](https://github.com/kloen)
+
+[Kloen](https://github.com/kloen) has put the time and work into Citra's Qt
+frontend to make it themeable. Now users can enjoy a dark mode and other custom
+colour schemes, just by editing a CSS stylesheet!
+
+{{< figure src="/images/entry/citra-progress-report-2017-august/theme-comparison.png" 
+    title="CHOOSE YOUR CHARACTER" alt="Comparison of Dark Theme and Light Theme" >}}
+
+## [Load Shared Font From System Archive](https://github.com/citra-emu/citra/pull/2784) by [wwylele](https://github.com/wwylele)
+
+**This does not eliminate the need for dumping a shared font from a legitimate system.**
+
+This in itself isn't an extremely visible or perceivable change from a user
+perspective, but it is something that helps us take another stride towards
+accurately recreating the way the 3DS actually operates. All system data is
+uniformly stored in the system archive now that the system font can be included
+with it. Citra does still fall back to the deprecated `shared_font.bin` file if
+it was dumped before, for compatibility's sake. `3dsutil` has been updated to be
+able to dump everything as a system archive, so that Citra may be able to work
+with it in a way that's more accurate to the actual hardware!
+
+## And Everyone Else
+
+Just because your work wasn't written on here doesn't mean it's not as important!
+Every little bit counts, every pull request inches the project one step closer
+down the long road of accurately emulating the Nintendo 3DS. So I would like to
+give my thanks to [everyone who's contributed](https://github.com/citra-emu/citra/graphs/contributors?from=2017-07-10&to=2017-08-30&type=c)
+these past two months, for giving us those extra metres down this road.