Looks like this was brought up there as a result of this article too, which is neat! And helpful since I was just messing with a node script that is heavily decoding base64
I like commits like this. Removes unnecessary work, easy to understand optimization, 40 lines gone 32 added so net smaller codebase, which usually means easier to maintain. Has ample comments and even uses one of my favorite tricks to do a ceiling function.
> Browsers recently added convenient and safe functions to process base 64 functions Uint8Array.toBase64() and Uint8Array.fromBase64()
Wow, finally! I've had to work around this so many times in the past (btoa/atob do not play nicely with raw binary data - although there are workarounds on the decode path involving generating data URIs)
Uhhh no, it's a huge net loss because the cost of sending it to the GPU and back greatly exceeds the cost of just doing it then and there in CPU; even on iGPU the kernel launch latency etc will kill it, and that's assuming the kernel build is free. Not to mention this is doing pow calls (!!), which is so ridiculous it makes me wonder if this was a kneejerk AI prompt.
Another post in this thread mentioned V8 sped this up by removing a buffer copy; this is adding two buffer copies, each about an order of magnitude slower.
Uhhh, go for it? You're welcome to link anything you like of course, but do you maybe want to address my actual points if you have any objections? Let's do some measurements, it sounds like you might be surprised by the outcome.
Web browser in a shader also sounds extremely inefficient, for obvious fundamental reasons.
Sorry, I was cracking a joke about the browser in a shader.
The GLSL I originally posted is from the "cursed mode" of my side project, and I use it to produce a data URI of every frame, 15 times per second, as a twisted homage to old hardware. (No, I didn't use AI :P )
That blog post left me hungry for more. I was expecting Daniel Lemire to provide a SIMD crazy optimized version that shows the default browser implementations are sub-optimal. But it's not in this article. Anyone knows?
I remember in the early days of Phoenix LiveView on an intranet app using http1 I noticed it was faster to base64 encode an image, putting it in an img tag and sending the diff through the Channel websocket than the regular http request through Cowboy.
No reason all browsers would not be able to be similar in performance eventually. Pleased this was noticed and being worked on by both v8 and Firefox team
Mozilla's "privacy" image prevents them from knowing what their browser actually does in the wild, while Google collects CPU time profiles from user devices, comprehensively, and hammers down the hotspots they find, and that refinement has been going on for many years.
Even if true (and I agree with sibling that I don't think that it is), base64 encoding/decoding feels like one of those things you'd have a micro benchmark for regardless. It's also shocking that the gap is so wide, as I feel like people working on such things would start with a fairly optimized v1.
I wonder if this is why Firefox feels so sluggish with some more complex SPAs.
That’s nonsense. Firefox has telemetry built in, it’s just that you can opt out of it. Your answer doesn’t explain why at all but instead just takes a wild guess at what might have happened. You don’t know if this was discovered in Chrome or in some other use of V8. Or maybe it was always fast in Chrome! What a weird non-answer.
Probably so you can put in line breaks? Seems common in base64 data, such as armored PGP keys or emails attachments. HTML attributes allow line breaks, although I haven’t seen it done for base64 images.
Now? There's no change. Also human readable text substantially consists of letters. But that's most of the base64 alphabet too. So this isn't like steganography. All the letters in the human-readable words are valid base64 characters too. The only thing about this is that you get to choose where to put the spaces and newlines. You can't exactly construct arbitrary payloads starting from arbitrary messages.
Maybe he means invisible whitespace characters that don't render? I haven't verified this but depending on the definition of whitespace it's possible you can pass a base64 string and insert an arbitrary number of them. When decoded per spec they do nothing so nobody notices them. But if you can pass the base64 string through you can receive or verify the hidden message. Lots of reasons you might want to hide data in plain sight.
https://chromium-review.googlesource.com/c/v8/v8/+/7208125 v8 changed yesterday to avoid the temp buffer which will likely double the base64 decode speed.
Looks like this was brought up there as a result of this article too, which is neat! And helpful since I was just messing with a node script that is heavily decoding base64
I like commits like this. Removes unnecessary work, easy to understand optimization, 40 lines gone 32 added so net smaller codebase, which usually means easier to maintain. Has ample comments and even uses one of my favorite tricks to do a ceiling function.
> Browsers recently added convenient and safe functions to process base 64 functions Uint8Array.toBase64() and Uint8Array.fromBase64()
Wow, finally! I've had to work around this so many times in the past (btoa/atob do not play nicely with raw binary data - although there are workarounds on the decode path involving generating data URIs)
base64 is embarrassingly parallel. So just pipe it to the GPU:
HN user: Ah yes let me casually scribble down a tweet-sized base64 encoder that runs parallel on GPU.
Bravo, that is a thing of beauty.
Uhhh no, it's a huge net loss because the cost of sending it to the GPU and back greatly exceeds the cost of just doing it then and there in CPU; even on iGPU the kernel launch latency etc will kill it, and that's assuming the kernel build is free. Not to mention this is doing pow calls (!!), which is so ridiculous it makes me wonder if this was a kneejerk AI prompt.
Another post in this thread mentioned V8 sped this up by removing a buffer copy; this is adding two buffer copies, each about an order of magnitude slower.
Come on guys...
Don't make me upload my web-browser-in-a-GLSL-shader snippet
Uhhh, go for it? You're welcome to link anything you like of course, but do you maybe want to address my actual points if you have any objections? Let's do some measurements, it sounds like you might be surprised by the outcome.
Web browser in a shader also sounds extremely inefficient, for obvious fundamental reasons.
Sorry, I was cracking a joke about the browser in a shader.
The GLSL I originally posted is from the "cursed mode" of my side project, and I use it to produce a data URI of every frame, 15 times per second, as a twisted homage to old hardware. (No, I didn't use AI :P )
https://github.com/Rezmason/excel_97_egg
That said, is `pow(vec4(2),-vec4(2,4,6,0))` really so bad? I figured it'd be replaced with `vec4(0.25, 0.0625, 0.015625, 1.0)`.
That blog post left me hungry for more. I was expecting Daniel Lemire to provide a SIMD crazy optimized version that shows the default browser implementations are sub-optimal. But it's not in this article. Anyone knows?
He's already done that: https://github.com/simdutf/simdutf?tab=readme-ov-file#base64 (see also https://github.com/lemire/fastbase64 )
I remember in the early days of Phoenix LiveView on an intranet app using http1 I noticed it was faster to base64 encode an image, putting it in an img tag and sending the diff through the Channel websocket than the regular http request through Cowboy.
Huh. How many frames per second could it hit, do you think?
It was for a turn-based game and I didn't benchmark for that, but it was noticeably faster for my use case.
Now that I think of it I should have cached the base64 in ETS to be even faster :)
Does anyone know why Firefox/Servo are so slow compared to the rest?
A few big things and lots of small things.
Big performance wins recently optimizing some core operations:
https://bugzilla.mozilla.org/show_bug.cgi?id=1994067 https://bugzilla.mozilla.org/show_bug.cgi?id=1995626
which brings it near chrome performance without the new v8 optimizations.
Still more work to do, including avoiding extra copies just like v8, and exploring more simd etc. Generic slow items for toBase64 and fromBase64: https://bugzilla.mozilla.org/show_bug.cgi?id=2003299 https://bugzilla.mozilla.org/show_bug.cgi?id=2003305
extra copying of results: https://bugzilla.mozilla.org/show_bug.cgi?id=2003461 https://bugzilla.mozilla.org/show_bug.cgi?id=1996197
No reason all browsers would not be able to be similar in performance eventually. Pleased this was noticed and being worked on by both v8 and Firefox team
Thanks for sharing!
Incredible that FF is even slower than a JS only implementation running in FF.
Mozilla's "privacy" image prevents them from knowing what their browser actually does in the wild, while Google collects CPU time profiles from user devices, comprehensively, and hammers down the hotspots they find, and that refinement has been going on for many years.
Even if true (and I agree with sibling that I don't think that it is), base64 encoding/decoding feels like one of those things you'd have a micro benchmark for regardless. It's also shocking that the gap is so wide, as I feel like people working on such things would start with a fairly optimized v1.
I wonder if this is why Firefox feels so sluggish with some more complex SPAs.
That’s nonsense. Firefox has telemetry built in, it’s just that you can opt out of it. Your answer doesn’t explain why at all but instead just takes a wild guess at what might have happened. You don’t know if this was discovered in Chrome or in some other use of V8. Or maybe it was always fast in Chrome! What a weird non-answer.
> However, when decoding, we must handle errors and skip spaces.
This had me scratching my head. Why would a base64 decoder need to skip spaces? But indeed, MDN documents this behavior:
> Note that: The whitespace in the space is ignored.
JS never ceases to surprise. Also, check out that typo :D
https://developer.mozilla.org/en-US/docs/Web/JavaScript/Refe...
Probably so you can put in line breaks? Seems common in base64 data, such as armored PGP keys or emails attachments. HTML attributes allow line breaks, although I haven’t seen it done for base64 images.
This might be for compatibility with XML Schema base64Binary, which collapses all whitespace (such as line breaks) to single spaces.
So technically it’s now possible to hide a payload in somewhat human-readable text, as long as it base64-decodes.
Now? There's no change. Also human readable text substantially consists of letters. But that's most of the base64 alphabet too. So this isn't like steganography. All the letters in the human-readable words are valid base64 characters too. The only thing about this is that you get to choose where to put the spaces and newlines. You can't exactly construct arbitrary payloads starting from arbitrary messages.
Maybe he means invisible whitespace characters that don't render? I haven't verified this but depending on the definition of whitespace it's possible you can pass a base64 string and insert an arbitrary number of them. When decoded per spec they do nothing so nobody notices them. But if you can pass the base64 string through you can receive or verify the hidden message. Lots of reasons you might want to hide data in plain sight.
What do you mean hide a payload?
Base64 isn’t obfuscation or encryption.