GTA III and Vice City fully reverse engineered: re3(github.com)
github.com
GTA III and Vice City fully reverse engineered: re3
https://github.com/GTAmodding/re3
163 comments
You would probably be pleasantly surprised with how easy handling modifications are in GTA:V.
The handling meta files are all XML now, in more or less plain English, and very well documented by the modding community.
I run an GTA:V multiplayer RPG server using FiveM[0] and the community around vehicle mods is really incredible. 3D modelers produce all kinds of real-world vehicles for GTA:V from Nissan GTR’s up to hyper-accurate law enforcement Dodge Chargers[1] with accurate Whelen lights, Setina bull bars, etc.
[0] https://fivem.net/ [1] https://redneckmods.com/product?id=3fbaf24e-7775-432e-87e1-f...
The handling meta files are all XML now, in more or less plain English, and very well documented by the modding community.
I run an GTA:V multiplayer RPG server using FiveM[0] and the community around vehicle mods is really incredible. 3D modelers produce all kinds of real-world vehicles for GTA:V from Nissan GTR’s up to hyper-accurate law enforcement Dodge Chargers[1] with accurate Whelen lights, Setina bull bars, etc.
[0] https://fivem.net/ [1] https://redneckmods.com/product?id=3fbaf24e-7775-432e-87e1-f...
Haha, you weren't the only one doing this. I used to alter the fire engine to also have some absurd weight but and engine powerful enough to still move it. Their was also a time scaling option somewhere which would slow the game down and put it into a "matrix"-esque mode.
I used to jump into the fire engine, get on the wrong side of the road on a freeway and activate the slow motion before driving head on into the traffic. Their was something mesmerising about smashing your way through the cars and seeing them lift off into the air and fly into the buildings on either side of the freeway.
Eventually the police would show up but what could they do against a fire engine that was as dense as a sky scraper!
I used to jump into the fire engine, get on the wrong side of the road on a freeway and activate the slow motion before driving head on into the traffic. Their was something mesmerising about smashing your way through the cars and seeing them lift off into the air and fly into the buildings on either side of the freeway.
Eventually the police would show up but what could they do against a fire engine that was as dense as a sky scraper!
Yes now I remember the time changing too! That was so much fun.
Another fun thing I now remember was to turn the dampening on the Taxi’s suspension to a positive value, so if you even gave them the smallest possible knock they would gradually start to wobble like a jelly and then start bouncing and bouncing until they were leaping up higher than buildings.
Another fun thing I now remember was to turn the dampening on the Taxi’s suspension to a positive value, so if you even gave them the smallest possible knock they would gradually start to wobble like a jelly and then start bouncing and bouncing until they were leaping up higher than buildings.
If I remember correctly the file called "handling.cfg" and it was a huge unreadable CSV table :) good times.
Yeah that rings a bell!
Really couldn’t believe my luck when I found something so obvious (I still felt like I had mad hacker skills)
Really couldn’t believe my luck when I found something so obvious (I still felt like I had mad hacker skills)
Yes. It was a csv table with all the configs for physics engine for all vehicles.
Not exactly CSV but close.
The good thing is now you get to see the...less than optimal code that reads this file: https://github.com/GTAmodding/re3/blob/master/src/vehicles/H...
Yes ! I have a similiar memory. When I was a kid, My parents did not like me playing the game for many reasons, one being the in game prostitutes . I simply changed a file to have them removed .
We modded an SUV to have monster truck tires. It allowed you to drive over passenger vehicles like speed bumps.
Opened this thread expecting to share this fond gaming memory. I've wished that modern games was more like this more than once. Finding jumps for crossing entire maps was incredibly liberating!
> We turned the Karuma [...]
It was Kuruma, in Japanese it means just a "car".
It was Kuruma, in Japanese it means just a "car".
haha, funny, and thanks for the correction!
I remember raising sea level in San Andreas.
Weapon damage was also controllable. I accidentally discovered a hack I used in the multiplayer (MTA mod) that made me receive 0 damage from non-projectile weapons.
Weapon damage was also controllable. I accidentally discovered a hack I used in the multiplayer (MTA mod) that made me receive 0 damage from non-projectile weapons.
As a basic web / JS developer I’m envious that creators of this repo get to work on awesome methods and classes called “Crime.h” and “!ped->bRenderScorched && !ped->bBodyPartJustCameOff”.
Beats “div.container” in my book.
Beats “div.container” in my book.
I get you. I was a game developer a couple of lifetimes ago. I once had to investigate and fix a bug called "HANDBAG TURNS INTO A BANANA". Never had anything like that in the corp world :_(
Hang in there. We once had a bug "PRODUCT IMAGE TURNS INTO A DOG" due to some code that kept falling back to an embarrassing placeholder.
In my very first developer job, I was creating a backend for a social media-type phone app. This was sort of a skunk works project that the founder and "Chief Innovation Officer" of a successful company wanted to do on the side, but he only had the budget for a couple of juniors.
Anyway, I didn't know a lot of things, so at one point I got the test environment and production environment tangled up in the code. I don't even remember the details of this confusion and I'm not sure I'd be capable of understanding how this was possible, but the long and the short of it is that one time the boss came down and asked something like "Why am I a horse?", and the answer was that he was being logged in as a user on the test environment which had a name that was a horse pun and a horse for a profile picture. Fun times.
Anyway, I didn't know a lot of things, so at one point I got the test environment and production environment tangled up in the code. I don't even remember the details of this confusion and I'm not sure I'd be capable of understanding how this was possible, but the long and the short of it is that one time the boss came down and asked something like "Why am I a horse?", and the answer was that he was being logged in as a user on the test environment which had a name that was a horse pun and a horse for a profile picture. Fun times.
Had 'under strained conditions, debug console comments negatively about your mother.'
A debug line someone thought wouldn't ever see the light of day.
A debug line someone thought wouldn't ever see the light of day.
I think "Right-click, see spiderman" was the highlight of my career.
"Title name says POOP" cca. 2006
Reminds me of the interview with Tarn Adams from Dwarf Fortress.
https://www.youtube.com/watch?v=VAhHkJQ3KgY
https://www.youtube.com/watch?v=VAhHkJQ3KgY
is it more stressful working on games than in the corp world?
I have never had a job that wasn't at least games adjacent, but I expect the comparison is not that clear. There is a huge diversity of commercial game development, just as there is for commercial software development generally. On average game development is probably more stressful because: it is an entertainment product in addition to being a piece of software; the funding models for independent game developers are often project based, which can lead to a lot of instability.
There are corners of game development that are more "chill" than others. Studios doing work-for-hire for other projects can build sustainable businesses. People in the "serious" game and simulation space seem to have a pretty normal work experience. Likewise shared tech teams at the biggest publishers/developers are more insulated from business variability and shifting project deadlines.
There are corners of game development that are more "chill" than others. Studios doing work-for-hire for other projects can build sustainable businesses. People in the "serious" game and simulation space seem to have a pretty normal work experience. Likewise shared tech teams at the biggest publishers/developers are more insulated from business variability and shifting project deadlines.
Ask CDPR developers.
It seems like bigger game companies push for lots of crunch and are more stressful, while smaller companies and indie dev teams are more relaxed.
This is all anecdotal from interviews, news, etc that I've seen.
This is all anecdotal from interviews, news, etc that I've seen.
of course it's going to vary, but yes.
I don't know if the new development model of "hit release date and fix everything after we sell the DLC" has made this better or worse.
I don't know if the new development model of "hit release date and fix everything after we sell the DLC" has made this better or worse.
[deleted]
You can still write a fully working page with a body but without a head, and a parent can do whatever it wants with their children, destroy them if necessary... It's not that bad...
A lot of my frontend code should probably live in a file called crime.js
Great collection of real video game changelog entries: https://twitter.com/thestrangelog
Then you'll love "CCutsceneMgr::RemoveEverythingFromTheWorldForTheBiggestFuckoffCutsceneEver()" - yes, that is a name originally chosen by R*!
Quirky variable names does not a good code base make.
It’s not a quirky variable name, it’s just the description of what’s happening in the game.
Rockstar forgot to strip the PlayStation binaries so I suppose those are the variable names of the original codebase.
Those are struct names though. So they wouldn't be in there. They did share some headers with the MTA developers though and that's where we got bRenderScorched from. i think the body part name is ours though.
Wow... how did mta convince rockstar to share stuff like that? For what purpose?
R* was a lot more open at the time and I guess they wanted to help them implement a multiplayer mode. that was before the whole hot coffee story.
Guess it was good market research.
Now GTA V / GTA Online made half a billion dollars in 2019, six years after it was released.
Now GTA V / GTA Online made half a billion dollars in 2019, six years after it was released.
Definitely. Early GTA Online was heavily inspired by MTA game modes.
For example, the races on crazy roads floating in the sky was a hallmark of MTA. It's far too abstract to think R* came up with that idea independently of the many years of prior art in MTA.
It's a shame they only copied the game modes and not the architecture, as GTA Online's biggest downfall is the P2P networking.
For example, the races on crazy roads floating in the sky was a hallmark of MTA. It's far too abstract to think R* came up with that idea independently of the many years of prior art in MTA.
It's a shame they only copied the game modes and not the architecture, as GTA Online's biggest downfall is the P2P networking.
[deleted]
Maybe this site is the best place to ask. There are two major things we could use help with:
- the games do not run particularly well on the raspberry pi (any version really) and I have a feeling it's at least partially due to the OpenGL layer. We could really use someone with how to optimize OpenGL (especially on the pi).
- physics have always been wonky beyond 30fps. I think someone with a good knowledge of game physics could really help us here.
- the games do not run particularly well on the raspberry pi (any version really) and I have a feeling it's at least partially due to the OpenGL layer. We could really use someone with how to optimize OpenGL (especially on the pi).
- physics have always been wonky beyond 30fps. I think someone with a good knowledge of game physics could really help us here.
I did some of the physics work (but was not the lead) for Papyrus racing games in the ‘95-‘97 era. Email’s in my profile if I can help.
My recollection of Vice City is that the vehicle dynamics model is not particularly realistic but optimized for “fun gameplay with a hint of realism” whereas Papyrus optimized towards “realistic gameplay with a hint of fun”. :D
Our physics engine was deeply, deeply tied to a particular isochronous rate (30 Hz, then later 60Hz, IIRC) with all kinds of hackery to make that happen and divorce rendering rate from physics rate. (On the PSX, we ran physics engine tied to the VBlank.)
My recollection of Vice City is that the vehicle dynamics model is not particularly realistic but optimized for “fun gameplay with a hint of realism” whereas Papyrus optimized towards “realistic gameplay with a hint of fun”. :D
Our physics engine was deeply, deeply tied to a particular isochronous rate (30 Hz, then later 60Hz, IIRC) with all kinds of hackery to make that happen and divorce rendering rate from physics rate. (On the PSX, we ran physics engine tied to the VBlank.)
Yeah, i think so too. In Vice City the most surprising thing to me was that they actually modelled tire temperature. it's probably a bit crude but i found it interesting.
As for the physics code, there's a big ProcessControl function all sorts of physical entities that does everything that has to happen to that entity each frame. some of it is physics, some not.
My theory as to why varying timestep produces different results is that the physics calculations form some sort of differential equations whose solution depends on the timestep. That may be wrong but it makes sense to me at least.
My approach to fixing it would be to log all sorts of forces applied to the physical objects each frame and log them so they can be compared. this is quite a bit of stuff to implement though so nobody has done it yet. if you or somebody is interested in that, we'd appreciate that a lot.
Ha, back in that same era I was working at a small game company that was in discussions with Papyrus about doing some work on the Indy Car Racing game - probably a Playstation port. It never happened - I think it was determined that it wouldn't sell enough to cover the development costs. I don't remember much else except we played the PC game ("research") and it was super difficult - you needed a lot of skill not to crash at every curve. I also seem to recall that the polygon models for the cars were based on manually edited text files. Good times.
> We could really use someone with how to optimize OpenGL (especially on the pi).
Not sure that gonna help much.
I think what you should do instead, port your OpenGL code to OpenGL ES. Specifically, Raspberry Pi 1-3 support GLES 2.0, Pi 4 supports GLES 3.1. As a nice side effect, renderer will become portable to smartphones.
This may or may not be a lot of work, depending how exactly you use OpenGL.
Pi 4 introduced Vulkan as well, but unlike GLES I have not tested it and not sure how good is the support.
Not sure that gonna help much.
I think what you should do instead, port your OpenGL code to OpenGL ES. Specifically, Raspberry Pi 1-3 support GLES 2.0, Pi 4 supports GLES 3.1. As a nice side effect, renderer will become portable to smartphones.
This may or may not be a lot of work, depending how exactly you use OpenGL.
Pi 4 introduced Vulkan as well, but unlike GLES I have not tested it and not sure how good is the support.
I have in fact tested everything with GLES 2.0, 3.1, and desktop GL 2.1 and 3.3 and they should all work. If it's as efficient as it could be is another question. I find OpenGL rather messy as an API compared to d3d9.
I don’t have GTA3 but I do have VC on Steam. Compiled 32-bit Windows build of VC with GL2. Here’s a renderdoc capture from my PC: http://const.me/tmp/vc-win10-gl.zip
1. On the main rendering pass, your code renders stuff mostly back to front. Consider sorting opaque objects by Z and render then front to back. If you do that, early Z rejection gonna save tons of pixel shaders and fill rate. The sorting doesn’t need to be perfect, an approximate will do as well, but with these ~5k draw calls I’m pretty sure even C qsort gonna be adequate on Pi4.
Translucent objects need to be rendered back to front like you’re currently doing.
2. In your pixel shader you often have this code:
Don’t do that. Write another pixel shader for cases when you don’t need alpha tests. GPUs disable some optimizations (early Z rejection is one of them) when you calling discard in GLSL/HLSL.
3. Up to EID 5328, the game rendered the next frame. Starting from EID 5377, the game was finishing rendering of the previous one. That’s a good idea by itself, the problem with that, the temporary texture has size 4096*4096 pixels. Only top-center 3840*2160 portion was actually used. My desktop PC is fast enough to deliver 60 FPS, but on RPi you should not use textures much larger than necessary. All modern GPUs support non-power-of-2 textures, Pi4 included.
4. You are using 2x MSAA for the main pass. At least for my test cases on Pi4 (see e.g. this project https://github.com/Const-me/Vrmac ), even 2x MSAA ruined performance, if you use the same setting on Pi4, I think that contributed the most to the performance issue.
Try to disable MSAA and see what gonna happen. If you’ll find out it helps with the performance but the quality is too bad now, try to implement a cheaper substitute somehow, search the web for FXAA and SMAA keywords.
1. On the main rendering pass, your code renders stuff mostly back to front. Consider sorting opaque objects by Z and render then front to back. If you do that, early Z rejection gonna save tons of pixel shaders and fill rate. The sorting doesn’t need to be perfect, an approximate will do as well, but with these ~5k draw calls I’m pretty sure even C qsort gonna be adequate on Pi4.
Translucent objects need to be rendered back to front like you’re currently doing.
2. In your pixel shader you often have this code:
if( a < u_alphaRef.x || a >= u_alphaRef.y ) discard;
Don’t do that. Write another pixel shader for cases when you don’t need alpha tests. GPUs disable some optimizations (early Z rejection is one of them) when you calling discard in GLSL/HLSL.
3. Up to EID 5328, the game rendered the next frame. Starting from EID 5377, the game was finishing rendering of the previous one. That’s a good idea by itself, the problem with that, the temporary texture has size 4096*4096 pixels. Only top-center 3840*2160 portion was actually used. My desktop PC is fast enough to deliver 60 FPS, but on RPi you should not use textures much larger than necessary. All modern GPUs support non-power-of-2 textures, Pi4 included.
4. You are using 2x MSAA for the main pass. At least for my test cases on Pi4 (see e.g. this project https://github.com/Const-me/Vrmac ), even 2x MSAA ruined performance, if you use the same setting on Pi4, I think that contributed the most to the performance issue.
Try to disable MSAA and see what gonna happen. If you’ll find out it helps with the performance but the quality is too bad now, try to implement a cheaper substitute somehow, search the web for FXAA and SMAA keywords.
Thank you for your input!
For shaders I'm never sure if I should do something dynamically or switch to a different shader, but it makes sense of course to kill the alpha test code if it's not used.
As for MSAA, our support for that is pretty rather young and it gives artifacts. I'm actually surprised it's on at all by default...
For shaders I'm never sure if I should do something dynamically or switch to a different shader, but it makes sense of course to kill the alpha test code if it's not used.
As for MSAA, our support for that is pretty rather young and it gives artifacts. I'm actually surprised it's on at all by default...
> it makes sense of course to kill the alpha test code if it's not used.
It’s used, but only for some meshes like vegetation, u_alphaRef was like [ 0.50196, 1000.00 ] for them.
> I'm actually surprised it's on at all by default
I don’t think I have adjusted anything, just built and run your code on Win10 with GL. This doesn’t mean it gonna use the same defaults on Pi, but it might.
It’s used, but only for some meshes like vegetation, u_alphaRef was like [ 0.50196, 1000.00 ] for them.
> I'm actually surprised it's on at all by default
I don’t think I have adjusted anything, just built and run your code on Win10 with GL. This doesn’t mean it gonna use the same defaults on Pi, but it might.
It's a bit of an underpowered GPU, but I would have thought it would be powerful enough to run an early PS2 era game like this.
It might just be really unhappy with the number of draw calls that you are doing. Looks like you are currently doing a drawcall per mesh; You might need to pack multiple meshes into a single drawcall.
Do you have any idea how many drawcalls per frame you are currently doing?
It might just be really unhappy with the number of draw calls that you are doing. Looks like you are currently doing a drawcall per mesh; You might need to pack multiple meshes into a single drawcall.
Do you have any idea how many drawcalls per frame you are currently doing?
Using renderdoc with normal opengl i count 660 draw calls on a hopefully representative frame.
Multiple meshes per drawcall are hard due to the architecture of renderware. I have considered sharing vertex and index buffers though. Do you think that will make a big difference?
I suspect that sharing vertex/index buffers won't really help (alone).
Last time I profiled something graphically intensive on the pi 4, I found that touching any of the draw state or uniforms between draw calls would trigger an expensive re-validation on the whole pipeline. 50% of my cpu time was spent in the driver.
Of course, the app I was profiling was pushing quite a lot of uniforms per draw call. Your workload might be different, so profile before doing anything drastic.
I recommend building a version of mesa with symbols so you can see exactly where CPU time is spent.
BTW, try prepending `mesa_glthread=true` to your command to see if that gives you a free speed boost.
Last time I profiled something graphically intensive on the pi 4, I found that touching any of the draw state or uniforms between draw calls would trigger an expensive re-validation on the whole pipeline. 50% of my cpu time was spent in the driver.
Of course, the app I was profiling was pushing quite a lot of uniforms per draw call. Your workload might be different, so profile before doing anything drastic.
I recommend building a version of mesa with symbols so you can see exactly where CPU time is spent.
BTW, try prepending `mesa_glthread=true` to your command to see if that gives you a free speed boost.
hm, state changes between draw calls are hard to get rid of...after all that's why they're separate calls in the first place. profiling the driver is probably indeed something i should look into.
You should be able to batch CPU-side drawcalls by packing multiple meshes into bigger vertex buffer, then generating draw parameters for every object into SSBO/UBO, and then utilizing ARB_multi_draw_indirect to draw everything at once. To change textures inside these batched drawcalls you might need ARB_bindless_texture. This talk is helpful: https://www.gdcvault.com/play/1020791/Approaching-Zero-Drive...
I don't know if these extensions are available on Pi driver, though.
I don't know if these extensions are available on Pi driver, though.
I am a developer but you guys are speaking a foreign language from my perspective in this thread. Where can I learn about graphical programming for the Pi, or are you guys well versed in game programming in general and applying it here? Sounds like an interesting skill set and challenge
https://antongerdelan.net/opengl/ is a decent resource for learning opengl (the author was a TA in my graphics course in college :D). And I do think it's worth Lear ing opengl over vulkan today. Vulkan dials the api complexity up to 11. If you're happy with windows only, then directx 11 would be a good choice too, but I don't have a good reference off hand.
The problem with graphics programming is that really you need to learn two things simultaneously: how 3d graphics are rendered, and how to use a GPU api. They can both be a significant thing to learn alone, and the complexity of cramming the two into your brain together makes things hard.
The problem with graphics programming is that really you need to learn two things simultaneously: how 3d graphics are rendered, and how to use a GPU api. They can both be a significant thing to learn alone, and the complexity of cramming the two into your brain together makes things hard.
Seconded, I too would like resources to learn more about this topic. Spend most of my time with business logic.
Try out an introductory OpenGL tutorial. There are many out ther. Just drawing a square will be a good lesson
> physics have always been wonky beyond 30fps
I suppose this is because the original developers knew exactly what hardware this would run on and the achievable framerate on the PS2 and so took lots of short-cuts in the code?
Edit: Incredible project by the way.
I suppose this is because the original developers knew exactly what hardware this would run on and the achievable framerate on the PS2 and so took lots of short-cuts in the code?
Edit: Incredible project by the way.
They actually did try to make it work at all frame rates but I don't think they were actually able to test it very well because the hardware, as you said, didn't do it.
A lot of games were hardcoded for a specific FPS. Even modern-ish ones!
Our local computer graphics prof hated this and set the loop rate random for the opengl idle function on our submission server. At one point it hang for 5 minutes between frames.
Brilliant move, must have been one hell of a smart guy.
I hate to say it, but fixed frame rates these days seem very 'lazy'? Obviously depending on use case and hardware.
I hate to say it, but fixed frame rates these days seem very 'lazy'? Obviously depending on use case and hardware.
I wouldn't call it lazy. All development is compromise. Especially game development.
If you know the game is only going to run on certain hardware, and you make the most money during the launch period of the game, it doesn't make sense to spend time, effort and money on something niche for the future.
If you know the game is only going to run on certain hardware, and you make the most money during the launch period of the game, it doesn't make sense to spend time, effort and money on something niche for the future.
Delta times make difficulty for games that rely on lockstep for multiplayer. It's why factorio, for example, is capped at 60fps and will slow down gameplay at slower framerates.
Time corrected vertlet integration[1] is really approachable and I'd strongly recommend trying it out.
[1]: https://www.gamedev.net/articles/programming/math-and-physic... Edit: http://archive.gamedev.net/archive/reference/articles/articl...
[1]: https://www.gamedev.net/articles/programming/math-and-physic... Edit: http://archive.gamedev.net/archive/reference/articles/articl...
Could box86 help here? https://github.com/ptitSeb/box86
No, according to README.md re3 works natively on arm64.
Can you give some more details how the reverse engineering was done exactly? (And maybe put that also in the README or somewhere, because I think this is probably interesting to some.)
You write:
> re3 was started sometime in the spring of 2018, initially as a way to test reversed collision and physics code inside the game. This was done by replacing single functions of the game with their reversed counterparts using a dll.
How exactly? So the game EXE exported all the function symbol names? How did you know the function signature (arguments, return type)? And then it is enough to preload a DLL to replace the function? There is no global state or so which must be accessed?
You write:
> re3 was started sometime in the spring of 2018, initially as a way to test reversed collision and physics code inside the game. This was done by replacing single functions of the game with their reversed counterparts using a dll.
How exactly? So the game EXE exported all the function symbol names? How did you know the function signature (arguments, return type)? And then it is enough to preload a DLL to replace the function? There is no global state or so which must be accessed?
Maybe you'll find the first commit instructive: https://github.com/GTAmodding/re3/tree/600bf0351476a5a21aabb...
We were very lucky that we had symbols for global things, which inludes function names and signatures but not return types. We then replaced the first instruction of a function with a jump to our own. global variables can be done with a reference to a raw address. virtual methods are a bit of a challenge but we found way to handle them.
We were very lucky that we had symbols for global things, which inludes function names and signatures but not return types. We then replaced the first instruction of a function with a jump to our own. global variables can be done with a reference to a raw address. virtual methods are a bit of a challenge but we found way to handle them.
Wow, that's cool, so you replaced each of the game's functions individually until you had the majority replaced and working stand-alone?
Not the majority. every single one!
(Except libraries like Renderware and Miles Sound System, because there'd be almost no point in wasting time on that. Only R* code was reversed.)
That's true, though I did write a replacement for RW. It's not an exact copy but it does the job.
Wow, impressive!
Can the assets be "reverse engineered" as well?
Also, can R* shut this project down? Is there a legal basis for them?
Can the assets be "reverse engineered" as well?
Also, can R* shut this project down? Is there a legal basis for them?
Apparently, this project requires the game assets to work - so you have to own a (legitimate) PC version of the game. https://github.com/GTAmodding/re3#intro
Of course, R*/Take-Two could always make a case.
Of course, R*/Take-Two could always make a case.
Assets are typically "reverse engineered" by writing tools that convert them to and from whatever proprietary formats the game uses. Other than that, there's not much else to do with them once they're in standard format.
As for the legality of the whole project: there's several different legal objections one could raise under copyright law:
1. That the EULA prohibits reverse-engineering
2. That the resulting reverse-engineering project is infringing the copyright on GTA3's program code (which it kind of has to in order to be a faithful disassembly)
As for the legality of the whole project: there's several different legal objections one could raise under copyright law:
1. That the EULA prohibits reverse-engineering
2. That the resulting reverse-engineering project is infringing the copyright on GTA3's program code (which it kind of has to in order to be a faithful disassembly)
Not sure what reverse engineering assets means. There is source code for them so they are what they are.
With "asset reverse engineering" I meant: recreating the assets from scratch.
Just like a painter reproducing another painting.
[deleted]
I think that for games like those, which more or less defined entire generations of people (the ones that were into gaming ofc) in a lot of countries, it should be mandatory that their source code is made public after like, ten years or something.
Sure, a company created it and it should be its intellectual property for enough time to return the investment etc. But something ingrained so much into people imo trancends that and should be considered belonging to the public realm.
Sure, a company created it and it should be its intellectual property for enough time to return the investment etc. But something ingrained so much into people imo trancends that and should be considered belonging to the public realm.
I can imagine a policy mandating that for all commercial software, source code goes into escrow for automatic release 10 or 20 years later.
The company doesn't have to be around anymore, rights don't have to be negotiated, etc.
Short of that I wish more companies would do it voluntarily. I'm not into id games, but they're one example that comes to mind.
The company doesn't have to be around anymore, rights don't have to be negotiated, etc.
Short of that I wish more companies would do it voluntarily. I'm not into id games, but they're one example that comes to mind.
This is critical imo, if only because software are cultural artifacts that should be able to be archived just like any other cultural artifacts.
Have you ever heard of free software? ;)
I have never heard of compulsory source releasing
Any law in the end requires a violent threat. This suggestion implies that someone who does not publish their source code should, after som ignored formal letters, get armed men sent to their office to kidnap whoever is responsible.
All nice sounding laws, such as gov funded welfare, gov funded hospitals, gov funded schools, and the endless regulations, implicitly carry a threat of violence, and when judging the ethics of a carrying a law out, that violent threat must be included. Government is firstly about guns.
All nice sounding laws, such as gov funded welfare, gov funded hospitals, gov funded schools, and the endless regulations, implicitly carry a threat of violence, and when judging the ethics of a carrying a law out, that violent threat must be included. Government is firstly about guns.
Quite the slippery slope fallacy.
You're also conflating civil and criminal law. Copyright is civil law. The government doesn't send the FBI in to ensure some publisher respects that a book's copyright has lapsed.
You're also conflating civil and criminal law. Copyright is civil law. The government doesn't send the FBI in to ensure some publisher respects that a book's copyright has lapsed.
Of course there is a violent threat behind copyright law as well - enforcing the right to ownership or the right to copy. Why else do you think anyone complies with laws, or cares what government decides? Government is about violent force
Check the other branches out! There is one for GTA Liberty City Stories[1], and another one for GTA Vice City[2].
This project enabled people to port GTA to the PS Vita device[3]. This device hasn't seen any games from the franchise, so this is a big thing.
[1] https://github.com/GTAmodding/re3/tree/lcs
[2] https://github.com/GTAmodding/re3/tree/miami
[3] https://github.com/TheOfficialFloW/re3
This project enabled people to port GTA to the PS Vita device[3]. This device hasn't seen any games from the franchise, so this is a big thing.
[1] https://github.com/GTAmodding/re3/tree/lcs
[2] https://github.com/GTAmodding/re3/tree/miami
[3] https://github.com/TheOfficialFloW/re3
Nice labor of love, and team accomplishment. I might have to re-buy Vice City (for game assets), just to try this on my Linux GPU box.
I got into the GTA franchise when (after not playing video games for a long time) I happened to see an old PS2 Slim that someone had put out beside the trash on the curb, brought it home, and started buying old games on eBay for a few dollars.
Adolescent humor and terrible themes&glorification aside, GTA Vice City led me to GTA San Andreas, which was both a compelling open world, and some great gameplay. I came to very much dislike members of other gangs, who were always starting crap when I was just going about my business -- which I suppose fit the story, and was also good mechanics for a game.
At some point, I had to buy a PS3, for GTA IV, which had been out a while. The PlayStation was a GTAstation. At the end of the story mode, as the cast-of-thousands credits rolled, over a montage of views of the New York City world they'd built, my reaction was that this game was a significant achievement for humankind.
And it was engaging enough, that, by endgame time, er, I might've developed muscle memory for consistent PVP headshots, high-speed "chicken" drive-bys, and literally driving circles around another player's car while concentrating fire on their driver position. One of the highest compliments you can receive is being wrongly accused of cheating.
In the very limited time I have for gaming lately (none, at the moment), it's usually been Ubisoft open worlds, but, whenever I check back into GTA Online freeroam, I'm reminded that it's the most actively fun for me. Cycles of retribution with another player or three, for the most creative or spectacular ways of fighting each other. And sometimes I even have the personally fulfilling opportunity to teach-- OP Mk. II griefers, that the Buzzard is still to be respected. (But forget heist stealth missions with randoms... :)
(Story-wise alone, The Last of Us might be my all-time favorite game. I dislike zombie stories in any medium, but, by the climax, I was misty-eyed, caring for the child character I was protecting.)
I got into the GTA franchise when (after not playing video games for a long time) I happened to see an old PS2 Slim that someone had put out beside the trash on the curb, brought it home, and started buying old games on eBay for a few dollars.
Adolescent humor and terrible themes&glorification aside, GTA Vice City led me to GTA San Andreas, which was both a compelling open world, and some great gameplay. I came to very much dislike members of other gangs, who were always starting crap when I was just going about my business -- which I suppose fit the story, and was also good mechanics for a game.
At some point, I had to buy a PS3, for GTA IV, which had been out a while. The PlayStation was a GTAstation. At the end of the story mode, as the cast-of-thousands credits rolled, over a montage of views of the New York City world they'd built, my reaction was that this game was a significant achievement for humankind.
And it was engaging enough, that, by endgame time, er, I might've developed muscle memory for consistent PVP headshots, high-speed "chicken" drive-bys, and literally driving circles around another player's car while concentrating fire on their driver position. One of the highest compliments you can receive is being wrongly accused of cheating.
In the very limited time I have for gaming lately (none, at the moment), it's usually been Ubisoft open worlds, but, whenever I check back into GTA Online freeroam, I'm reminded that it's the most actively fun for me. Cycles of retribution with another player or three, for the most creative or spectacular ways of fighting each other. And sometimes I even have the personally fulfilling opportunity to teach-- OP Mk. II griefers, that the Buzzard is still to be respected. (But forget heist stealth missions with randoms... :)
(Story-wise alone, The Last of Us might be my all-time favorite game. I dislike zombie stories in any medium, but, by the climax, I was misty-eyed, caring for the child character I was protecting.)
How possible is it to use the PS2 version for extracting assets? If the answer is "do this" or "we don't know", I am (in principle) willing to spend some time looking into it, but if the answer is "it won't work", that is useful information as well.
Hi, (co-)author here. Cool to see this here :)
>Do not use features from C++11 or later.
Is this a technical requirement to have compiler feature parity on all platforms?
Is this a technical requirement to have compiler feature parity on all platforms?
Pretty much, but also our goal is to recreate the code in a way that R* could have plausibly written it. And that implies no C++11.
To add to what aap mentioned, it also made it relatively easy to compile re3 and reVC with the original IDE it was written in - CodeWarrior 7. Some things accidentally made it in, like sized enums, but overall no major surgery required, or rather, far less than I expected.
This is fascinating, thanks!
Can you speak about what it took to achieve this? (required knowledge, process, etc...)
Can you speak about what it took to achieve this? (required knowledge, process, etc...)
We were lucky that we had symbols from PS2 gta3 and the android games. other than that it was a lot of reading code in IDA and massaging it back into c++. I made a little video about part of the process [1] but i never did part 2.
The strategy for gta3 was to replace function by function of the game until we had everything replaced. for VC we evolved our existing code base by, again, reversing function by function until we had everything done. Just not by dll injection this time.
[1] https://www.youtube.com/watch?v=22BeuOOERLo
The strategy for gta3 was to replace function by function of the game until we had everything replaced. for VC we evolved our existing code base by, again, reversing function by function until we had everything done. Just not by dll injection this time.
[1] https://www.youtube.com/watch?v=22BeuOOERLo
Hey, whatever happened to openrw? I remember trying to get into that back in 2017 but looks like this project ended up superseding it.
Any ideas to make it usable under ATI RS600 on Linux with Mesa drivers?
I assume you have tested it and it doesn't work? Maybe the GPU is simply too old. I'm doing everything with shaders and it's possible they are just too much for this GPU. Supporting a wide range of hardware (especially old one that nobody has anymore), is not easy.
> I'm doing everything with shaders... upporting a wide range of hardware (especially old one that nobody has anymore), is not easy.
What about adding just an option (e.g. `re3 --no-shaders`) for may it possible run re3 even on older hardware by disabling most of decorative shaders?
What about adding just an option (e.g. `re3 --no-shaders`) for may it possible run re3 even on older hardware by disabling most of decorative shaders?
[deleted]
Thanks for your efforts.
One thing I like about these re projects: they allow porting to different systems by recompiling.
It is an interesting feat to run SDLPoP (re'd prince of persia) on modern systems.
It is an interesting feat to run SDLPoP (re'd prince of persia) on modern systems.
How is it possible to RE a whole videogame? What's the process to go from bits to C++?
the bits are cpu instructions that you can analyze the data flow to revert the instructions into c++ code, either manually or assisted with a decompiler.
https://en.wikipedia.org/wiki/Decompiler
https://en.wikipedia.org/wiki/Ghidra
https://ghidra-sre.org/
https://en.wikipedia.org/wiki/Decompiler
https://en.wikipedia.org/wiki/Ghidra
https://ghidra-sre.org/
You have a long list of machine language instructions. You know (roughly) what the output of the compiler looks like when translating from source code to machine language for various constructs like functions, if statements, etc. (a compiler course helps here). Some reverse engineering tools provide partial automation for that to assist in this deciphering. After that, it's just a lot of painstaking, grinding work.
[deleted]
Is it true that NPC cars are not checked for collision?
And collision checking only starts when something happens near?
This is sort of true, they are "on rails" (meaning no physics calculations are performed) until something collides with it, like a bullet, a car or a player.
At that point they are handed off to a more expensive process which includes checking physics each frame.
At that point they are handed off to a more expensive process which includes checking physics each frame.
Of all the systems that you guys reverse engineered here what do you think is the coolest? Ai/pathfinding, quest systems, large world rendering?
I think everybody did the thing he was interested in most. For me that was rendering, physics, vehicles, camera and ps2 specific stuff like VU0 collision detection.
Nice!
Btw: Is there a list with projects that got done due to the pandemic? :)
Oh no I’m so excited to try this out on Linux.
By the way i just want to thank you from the bottom of my heart for reverse engineering it.
Now i don’t have to use Wine to run my childhood game on Linux
Now i don’t have to use Wine to run my childhood game on Linux
This is very very impressive ! This reminds me the efforts to reverse-engineer Super Mario 64 that led to SM64 PC (and looking forward to the Zelda Ocarina of Time reverse!)
Random question from a web developer who has zero knowledge of C++ or game development: I see no tests in this repo. Is this normal, or am I missing something?
Game designer checking in.
Unit tests are very rarely used in games. This is partially because they are so interactive, and partially because it would take a lot of time away from something else. The video game industry has always been under severe crunch. Why would you waste time writing something to check for a bug rather than fixing a different bug or adding a cool new feature? It’s not the healthiest approach but it was inevitable now that we can patch games at any moment.
Here is an example of a game called Factorio running its own unit tests. It’s a lot easier for this game as its deterministic and doesn’t need player interaction to function. https://youtu.be/LXnyTZBmfXM
There have also been attempts to use machine learning agents that act as human players. It’s going to be a while but the idea is that it would automate playtests which are used to hunt down the same bugs. https://youtu.be/ZZsSx6kAi6Y
Here is an example of a game called Factorio running its own unit tests. It’s a lot easier for this game as its deterministic and doesn’t need player interaction to function. https://youtu.be/LXnyTZBmfXM
There have also been attempts to use machine learning agents that act as human players. It’s going to be a while but the idea is that it would automate playtests which are used to hunt down the same bugs. https://youtu.be/ZZsSx6kAi6Y
I had a short stint at a AAA games company and I worked on one of the games. It was all written in C++ and we had absolutely 0 unit tests. All the testing was done by an army of QA guys.
That is wildly interesting. Every SaaS startup doing some 100k of $$$ in ARR will probably have long testing suites. AAA games making possibly tens of millions. 0 tests.
Lots of things are at play here (excuse the bad pun):
* SaaS stuff can be built very quickly (if you use cached node_modules or PHP composer's vendor directory) and deployed to parallel CI test runners that don't have much requirements. In contrast, your typical game takes much longer to (re)compile and as soon as you touch anything that requires graphics, you need some sort of (beefy) GPU in the server. Anyone who has ever tried virtualization with PCIe passthrough knows what a pain that is to set up, not to mention NVIDIA explicitly disallowing to virtualize their GPUs.
* It's easy to do unit tests for simple entities and operations (e.g. collision detection, physics)... but games are full of global state mutated from across the place.
* For everything that tests visual (or, god forbid, audio) output, computers are bad at testing. Change a texture? Great, now you have to re-check all your tests if they're still accurate and working. Humans are, for now, still better.
* Hardware-specific bugs: these are one of a goddamn nightmare. For most software, it behaves identically across computers - let's take most Docker containerized software... it will essentially run on anything that's amd64. But games integrate with OSes, especially the OS 3D stack (DX, OpenGL, Vulkan, Metal and on top of that the GPU drivers and GPU firmware) so incredibly deep that there are lots and lots of potential issues and workarounds. It's simply impossible to test for these.
* Closed source components in the loop: Games ship with all sorts of blobs, from video codecs to sound effect libraries. These are in your address space, but you can't really test them.
* Human tests are cheaper than writing test suites and setting up CI test farms (see the first point in my list).
* SaaS stuff can be built very quickly (if you use cached node_modules or PHP composer's vendor directory) and deployed to parallel CI test runners that don't have much requirements. In contrast, your typical game takes much longer to (re)compile and as soon as you touch anything that requires graphics, you need some sort of (beefy) GPU in the server. Anyone who has ever tried virtualization with PCIe passthrough knows what a pain that is to set up, not to mention NVIDIA explicitly disallowing to virtualize their GPUs.
* It's easy to do unit tests for simple entities and operations (e.g. collision detection, physics)... but games are full of global state mutated from across the place.
* For everything that tests visual (or, god forbid, audio) output, computers are bad at testing. Change a texture? Great, now you have to re-check all your tests if they're still accurate and working. Humans are, for now, still better.
* Hardware-specific bugs: these are one of a goddamn nightmare. For most software, it behaves identically across computers - let's take most Docker containerized software... it will essentially run on anything that's amd64. But games integrate with OSes, especially the OS 3D stack (DX, OpenGL, Vulkan, Metal and on top of that the GPU drivers and GPU firmware) so incredibly deep that there are lots and lots of potential issues and workarounds. It's simply impossible to test for these.
* Closed source components in the loop: Games ship with all sorts of blobs, from video codecs to sound effect libraries. These are in your address space, but you can't really test them.
* Human tests are cheaper than writing test suites and setting up CI test farms (see the first point in my list).
Because unit tests are usually worthless, except in some specific circumstances.
This. We write unit tests when we do math on financial transactions, clear cut migrations of data, and other very well-defined, high risk circumstances.
The other use-case for tests is explaining the expectation to new developers / teams under tight timelines. Example - we had a "surge" team come in - we were writing software that pulls data out of emails in hundreds of formats.
We set up infrastructure for it that says - here is your input, here is your ouput, here is 100 sample records, make unit tests pass for them. This worked amazingly well when you had a team writing RegEx, another team writing csv stuff, one team using ML, etc. Basically, the goal was "make it work, in 2 weeks, no one will ever touch this code again." Btw, the latter is ALWAYS, ALWAYS a lie - if a company sinks a few hundred thousand into a project, you WILL touch it again.
Anyway, this was more of a rant - basically unit tests ARE useless 90% of the time. Unit tests in Angular that are basically integration tests are absolutely disgusting.
Keep in mind that if you have coverage, any time you modify a class, you are now fixing a bunch of broken tests.
I guess that's why I am for testing individual methods and SRP.
The other use-case for tests is explaining the expectation to new developers / teams under tight timelines. Example - we had a "surge" team come in - we were writing software that pulls data out of emails in hundreds of formats.
We set up infrastructure for it that says - here is your input, here is your ouput, here is 100 sample records, make unit tests pass for them. This worked amazingly well when you had a team writing RegEx, another team writing csv stuff, one team using ML, etc. Basically, the goal was "make it work, in 2 weeks, no one will ever touch this code again." Btw, the latter is ALWAYS, ALWAYS a lie - if a company sinks a few hundred thousand into a project, you WILL touch it again.
Anyway, this was more of a rant - basically unit tests ARE useless 90% of the time. Unit tests in Angular that are basically integration tests are absolutely disgusting.
Keep in mind that if you have coverage, any time you modify a class, you are now fixing a bunch of broken tests.
I guess that's why I am for testing individual methods and SRP.
While I totally agree with you, I also wonder if unit testing is used in the engine itself.
That's because in the games industry there is no expectation of a v 2.0 (most games never sell well enough to warrant it) and therefore little concern about mantainability or the future. It's hard enough to get the 1.0 release out the door without the studio running out of budget.
I'm pretty sure that Rockstar North would have some tests in place. The creators of the reverse engineered project, simply couldn't access these tests as they wouldn't be included in the build. (At least, I'd imagine).
Also: It may be interesting to note that GTA:V's leaked build logs featured some notes about an "auto play bot test failed at {mission}". I'm pretty sure they would have to have similar tests (although perhaps at a smaller scale) for older GTA's as well.
It's quite hard to test heavily interactive code like this. Not totally impossible but you have to question if it's worth the effort.
It may be interesting to note that GTA:V's leaked build logs featured some notes about an "auto play bot test failed at {mission}".
I'm pretty sure they would have to have similar tests (although perhaps at a smaller scale) for older GTA's as well.
Lots of games are throwaway code - you essentially start over for the next one. That's wildly inefficient of course, which is why the generic reusable parts have been factored out into huge engines like Unreal Engine, and those generally have some test coverage.
It's normal. It's incredibly difficult to unit test most things in games (graphics, sound, player controls). Also most game logic is pretty simple and it either works or it very noticeably doesn't.
That said, often game logic has some unit tests.
That said, often game logic has some unit tests.
Any new technics revealed for speedrun community? AFAIK, WR for GTA VC is 8 minutes and this run cannot be possible w/o reverse engineering.
I have just realised that this is not a leak of source code but a new implementation. This version definitely needs a new speedrun, I have contacted Marushko.
My favorite game.
GTA III, Vice City and San Andreas had mod allowing to play online (multiplayer) called "Multi Theft Auto" (for Vice City there was another one called VC:MP)
As far as I remember, the MTA project was started by a guy who initially tried to build a trainer by reverse engineering one of the GTA games, in the end he ended with a multiplayer mod. Fun times.
GTA III, Vice City and San Andreas had mod allowing to play online (multiplayer) called "Multi Theft Auto" (for Vice City there was another one called VC:MP)
As far as I remember, the MTA project was started by a guy who initially tried to build a trainer by reverse engineering one of the GTA games, in the end he ended with a multiplayer mod. Fun times.
Very impressive. Neat that they're even intending to get a PS2 build working.
Wonder if they plan to do further work on graphical improvements.
Wonder if they plan to do further work on graphical improvements.
PS2 build is a lot of work unfortunately and I'm afraid it is me who'll have to do it. Guess we'll see how far we'll get...
I've done some graphical improvements but we want to keep it mostly true to the original spirit. But yes, there are some things i would still like to improve upon a bit.
I've done some graphical improvements but we want to keep it mostly true to the original spirit. But yes, there are some things i would still like to improve upon a bit.
What is so different about the PS2 version? Now that you have the source code for PC, why is it a lot of work to port it to PS2?
By the way, this project is so cool. Thanks for making this!
By the way, this project is so cool. Thanks for making this!
Now someone must add multiplayer to this.
[deleted]
This is so cool, I can't wait to see a multiplayer adaption now that it's open-source! Oh please reverse engineer GTA:IV and bring it back to life as well
What is this premake binary, and where are the sources for it?
EDIT: you can just install premake from package management, and use that instead of the checked in binaries.
EDIT: you can just install premake from package management, and use that instead of the checked in binaries.
premake is something like cmake. The sources are at https://github.com/premake/premake-core.
The binaries in the repo are from their github actions builds, from whenever I last updated the premake binaries in re3: https://github.com/GTAmodding/re3/pull/945
The binaries in the repo are from their github actions builds, from whenever I last updated the premake binaries in re3: https://github.com/GTAmodding/re3/pull/945
Excellent work, I replay these on my iPad every few years. It would be great to get these on the Switch.
Do you use a controller? I’ve thought about trying one of the mobile ports but the idea of playing with a touch screen doesn’t seem viable. Is this solved somehow?
It’s an excellent experience in my opinion. The virtual D-pad is fine for driving. It’s not ideal for on foot gunplay, but serviceable and that’s really a side dish to the main course driving experience. In just one or two places in some of the games you get a bit stuck. In GtA3 there’s a challenge where you need to chuck a grenade through an upper story window that’s really tricky, for example, but apart from that it’s fine and IMHO these games are incredible value for money at their current prices.
Now, somebody implement multiplayer!
Can we spend a minute for paying respect a license?
How it is called?
I still play my PC CD Roms patched under wine so this is neat thanks!
This is amazing.
I hope someone takes on doing the same for GTA: Chinatown Wars
I hope someone takes on doing the same for GTA: Chinatown Wars
I’m just curious to know how do developers reverse engineer
[deleted]
We turned the Karuma into a physics defying monster. It was 1000x heavier than the tank, so as you drove down the road other cars just bounced off and flew over the horizon, or exploded on impact.
Yet it still did 0-100 in 0.1 seconds and could drive up walls.
One time we jumped it from the third island, over the middle island and landed on a skyscraper roof on the 1st island.
From there we spent about an hour dropping grenades on police cars and the army who had no way to reach us up there.
Fun times. Shame Twitch or YouTube didn’t exist back then to tell anyone about it!