Palette lighting tricks on the Nintendo 64
(30fps.net)209 points by ibobev a day ago
209 points by ibobev a day ago
Note that the N64 was designed by SGI, And seeing as how influential SGI was for 3d graphics, I sort of assume the reverse, that the n64 probably has the most standard hardware of it's generation. I would be vaguely surprised if there was not an opengl library for it.
However there is a large caveat, 1. you have to think of the system as a graphics card with a cpu bolted on. and 2. the graphics system is directly exposed.
Graphics chip architecture ends up being a ugly hateful incompatible mess, and as such the vendors of said accelerators generally tend to avoid publishing reference documents for them, preferring to publish intermediate API's instead. things like OpenGL, DirectX, CUDA, Vulcan, mainly so that under the hood they can keep them an incompatible mess(if you never publish a reference, you never have to have hardware backwards compatibility, the up side is they can create novel designs, the down side is no one can use them directly) so when you do get direct access to them, as in that generation of game console, you sort of instinctively recoil in horror.
footnote on graphics influence: OpenGL came out of SGI and nvidia was founded by ex SGI engineers.
> that the n64 probably has the most standard hardware of it's generation
The Reality Coprocessor (or RCP) doesn't look like any graphics cards that previously came out of SGI. Despite the marketing, it is not a shrunk down SGI workstation.
It approaches the problem in very different ways is actually more advanced in many ways. SGI workstations had strict fixed function pixel pipelines, but RCP's pixel pipeline is semi-programmable. People often call describe it as "highly configurable" instead of programmable, but it was the start of what lead to modern Pixel Shaders. RCP could do many things in a single-pass which would require multiple passes of blending on a SGI workstation.
And later SGI graphics cards don't seem to have taken advantage of these innovations either. SGI hired a bunch of new engineers (with experience in embedded systems) to create the N64, and then once the project was finished they made them redundant. The new technology created by that team never had a chance to influence the rest of SGI. I get the impression that SGI was afraid such low-cost GPUs would cannibalise their high-end workstation market.
BTW, The console looks most like a shrunk down 90s SGI workstation is actually Sony's Playstation 2. Fixed function pixel pipeline with a huge amount of blending performance to facilitate complex multi-pass blending effects. Though, SGI wouldn't have let programmers have access to the Vector Units and DMAs like Sony did. SGI would have abstracted it all away with OpenGL
------------------
But in a way, you are kind of right. The N64 was the most forwards looking console of that era, and the one that ended up the closest to modern GPUs. Just not for the reason you suggest.
Instead, some of the ex-SGI employees that worked on the N64 created their own company called ArtX. They were originally planning to create a PC graphics card, but ended up with the contract to first create the GameCube for Nintendo (The GameCube design shows clear signs of engineers overcompensating for flaws in the N64 design). Before they could finish, ArtX were bought by ATI becoming ATI's west-coast design division, and the plans for a PC version of that GPU were scrapped.
After finishing the GameCube, that team went on to design the R3xx series of GPUs for ATI (Radeon 9700, etc).
The R3xx is more noteworthy for having a huge influence on Microsoft's DirectX 9.0 standard, which is basically the start of modern GPUs.
So in many ways, the N64 is a direct predecessor to DirectX 9.0.
> The GameCube design shows clear signs of engineers overcompensating for flaws in the N64 design
I haven't programmed for either console. Which features show this in what sense?
The RCP was actually two hardware blocks, the RDP which as you say did the fixed function (but very flexible) pixel processing and the RSP which handled command processing and vertex transformation (and audio!).
The standard api was pretty much OpenGL, generating in-memory command lists that could be sent to the RSP.
However the RSP was a completely programmable mips processor (with simd instructions in parallel).
One of my favorite tricks in the RDP hardware was it used the parity bits in the rambus memory to store coverage bits for msss
Shadow of the Colossus... https://www.youtube.com/watch?v=xMKtYM8AzC8
The amount of indie N64 development happening right now is wild. The platform is flourishing.
The system has seen a dozen of its most popular games decompiled [1] into readable source files, which enables easy porting to PC without an emulator. It also enables a ton of mods to be written, many of which will run on the original hardware.
There are numerous Zelda fan remakes [2]. Complete games with new dungeons and storylines.
The Mario 64 scene is on fire. Kaze has deeply optimized the game [3], and is building his own engine and sequels. If you like technical deep dives into retro tech, his channel is literally golden.
Folks are making crazy demos for the platform, such as Portal [4], which unfortunately brought Valve's lawyers' attention.
Lost games, such as Rare's Dinosaur Planet [5], have leaked, been brought up to near production ready status, been decompiled, and have seen their own indie resurgence.
[1] https://wiki.deco.mp/index.php/N64
[2] https://m.youtube.com/watch?v=bZl8xKDUryI
[3] https://m.youtube.com/channel/UCuvSqzfO_LV_QzHdmEj84SQ
The whole channel is gold. He has dozens of deep dives like this: https://m.youtube.com/watch?v=DdXLpoNLywg
And his game and engine are beautiful: https://youtu.be/Drame-4ufso
We did similar palette-based lighting techniques in our shareware game in the 90s. Basically, arranging the VGA 256-color palette so that each color we supported would have a gradient of N shades of the color. Illumination within each color could then be easily altered by adding or subtracting color indices.
It blows my mind how genius these game engineers were. They dealt with so many limitations and created such imaginative and brilliant solutions.
Limitations demand and produce extraordinary creativity. That's the secret behind pico8 and Animal Well and so many amazing games.
I wish I didn't think of a significantly better architecture for my 2d-pixel-art-game-maker-maker this weekend. Now it'll be another month before I can release it :(
Only recently did we figure out how to make Mario64 run at 30fps.
Around the end of the PS2’s lifetime, some engine dev friends of mine figure out to do palletized spherical harmonic lighting on the PS2. That was pretty straightforward.
What was tricky was a separate technique to get real cubemaps working on the PS2.
Unfortunately, these came too late to actually ship in any PS2 games. The SH trick might have been used in the GameCube game “The Conduit”. Same team.
I always thought triace had shipped sh lighting on ps2 but maybe it was just a demo?
http://research.tri-ace.com/Data/Practical%20Implementation%...
While I'm really happy we have faster systems now, there was something fun about about having to subvert constraints in games, and so satisfying and lovely when you did it right.
HN folks are probably familiar with raster interrupts (https://en.wikipedia.org/wiki/Raster_interrupt) and "racing the beam." I always associated this with the Atari 800. You weren't "supposed" to be able to do stuff like https://youtu.be/GuHqw_3A-vo?t=33, but Display List Interrupts made that possible.
What I didn't know until recently was how much Atari 2600's games owed to this kinda of craziness: https://www.youtube.com/watch?v=sJFnWZH5FXc
It's stuff like this that makes me think that if hardware stopped advancing, we'd still be able to figure out more and more interesting stuff for decades!
Demo scene and work like this is impressive. Yet I can't help but notice that it tends toward simpler more empty scenes. The kind of stuff one might expect in the background or as only a part of a game mechanic. It's as if there's just not enough resources to really make complete experiences with most of the techniques.
What I find more impressive are efforts like FastDoom or the various Mario-64 optimization projects which squeeze significantly better performance out of old hardware. Sometimes even while adding content and features. Maybe there is a connection between demo sceners and more comprehensive efforts?
I miss the PS1 and PS2 optimization. Most of them look amazing uprezzed to 1080p or 4k or more with emulation. Halo 2 era graphics in 4k is all we need imo. Yes that one is xbox but try Halo MCC Halo 2 in classic graphics. Still looks incredible.
GT3 heatwave summarizes it well.
"I showed a demo of GT3 that showed the Seattle course at sunset with the heat rising off the ground and shimmering. You can’t re-create that heat haze effect on the PS3 because the read-modify-write just isn’t as fast as when we were using the PS2. There are things like that."
https://old.reddit.com/r/ps2/comments/1cktw88/gran_turismos_...
https://youtu.be/ybi9SdroCTA?t=4103
It's not trying to emulate a real heatwave as new engines like UE5 does, that just tanks fps. It does "tricks" to do it instead. And honestly, looking at RTX tanking frame rates, I would rather have these cheap tricks.
A 299MHz MIPS runs this:
Shadow of the Colossus... https://www.youtube.com/watch?v=xMKtYM8AzC8
GoW2 https://youtu.be/IpKLwIIdvuk?si=TjifKmlYsUuvhk0F&t=970
FFXII https://youtu.be/NytHoYOs_4M?si=jE1Fxy40khEvV6Bn&t=51
GT4 https://www.youtube.com/watch?v=F6lZIxk_h9g (THE BOOTSCREEN crying)
Black (Renderware was a crazy engine) https://youtu.be/bZBjcwyq7fQ?si=Pev5ifpksJm4X6Oi&t=356
Valkyrie profile 2 https://youtu.be/9ScjO4NuUtA?si=Z29cR-hLsT2pnP2I&t=38
Rouge Galaxy https://youtu.be/iR1evzyl-7Q?si=fldm3-NnuFxOITMn&t=624
Burnout 3 https://www.youtube.com/watch?v=_r5r0nE1sA4
Jak and Daxter, Ratchet.
For GC - RE4, Metroid, The Zeldas... ofc. Looks crazy good.
I kneel.
With the PS2 you are right. With the PSX... so-so. Yes, it could match maybe a Pentium 90 almost 100, but a MMX pentium with 3DFX would stomp it and be on par of the N64 if not better.
MIPS CPU's are amazing, they can do wonders at low cicles. Just look at the PSP, or the SGI Irix.
Also, the PS2 "GPU" is not the same as the R4k CPU. BTW, on the PS2... the Deus Ex port sucked balls against the PC port, it couldn't fully handle the Unreal engine.
Yes, the PS2 did crazy FX, but with really small levels for the mentioned port; bear in mind DX was almost 'open word' for a huge chunk of the game.
> With the PSX... so-so. Yes, it could match maybe a Pentium 90 almost 100, but a MMX pentium with 3DFX would stomp it
Pentium much faster than MIPS CPU for game logic, 3dfx 50 MPixels/s fillrate matches Playstations 60 MPixels/s, Pentium FPU tho is no match for Playstation GTE 90-300K triangles per second meaning you would have to rely on CPU power alone for geometry processing (like contemporary Bleem) resulting in 166-233MHz Pentium minimum requirements. MMX would be of no help here, it was barely used in few games for audio effects.
Bleem it's an emulator; it emulates the architecture, is not a virtualizer. 233 MHZ to emulate the 33 MHZ PSX seems reasonable, Windows 95/98 take up a good chunk of the CPU themselves. But, you forgot something.
The PSX "GPU" just worked with integers and that's it. Any decent compiler such as GCC and flags like -ffast-math would emulate the both dead simple MIPS CPU and the fixed point GPU where no floats are used at all while taking tons of shortcuts. MMX? Ahem, MPEG decoding from videos. If you did things right you could even bypass the BIOS decodings and just call the OS MPEG decoding DLL's (as PPSSPP does with FFMPEG) and drop the emulation CPU usage to a halt and let your media player framework do the work for you.
I still think halo 3 looks a lot better than some modern games. Stuff like blur bloom and all that grass and foliage pop in does not in fact look good. It looks worse than just turning all of that off. And I can’t appreciate a high polygon count model when the game is a high speed fps so whats the point of that either. Halo 3 texture resolution to my eye is fine. I don’t think I would notice twice or 4x the size textures. Only thing I notice is the hardware demands.
It's very impressive to see "realistic" graphics on the N64. The demo reminds me of "ICO" for the PS2.
I've always wondered if it would be possible to create an SDK to abstract the N64 graphics hardware and expose some modern primitives, lighting, shading, tools to bake lighting as this demo does, etc. The N64 has some pretty unique hardware for its generation, more details on the hardware are here on Copetti.org:
https://www.copetti.org/writings/consoles/nintendo-64/