Pushing Java to the Limits: Processing a Billion Rows in under 2 Seconds by Thomas Wuerthinger

54 года назад

5,996 Просмотров

Last January a challenge was posted online by Gunnar Morling:How fast can you parse a file with one billion rows of weather data using Java?Little did we know this deceivingly simple question would lead us down a path that covered: parallelism, memory mapped files, SWAR techniques (SIMD as a register), bit twiddling, branchless code, mechanical sympathy, Graal native compilation and finally... turning to the dark side: using sun.misc.Unsafe.Join Thomas and Roy during this deep dive where they explain all the improvements and tricks that enabled them to go from a >4 minute reference implementation to processing a billion rows in under two seconds.Who knew Java could be _this_ fast?

Скачать видео

Комментарии:

@Workaholic42 - 09.10.2024 21:08

Great work and great presentation!

Ответить

@xsmile8874 - 10.10.2024 09:03

Greak work, Thank your.

Ответить

@jaime7295 - 11.10.2024 19:21

It was a great challange, it was a lot of fun. I hope there is next one

Ответить

@RoyWasse - 13.10.2024 20:54

I take my hat off for all the smart optimizations. Shows that understanding on a deep level how the OS, CPU, memory, compiler and language work matters a lot. And that creativity goes a long way.

Ответить

@zahidahmed7043 - 24.10.2024 22:45

The time spent on... worth it . kudos to those all amazing programmers.

Ответить

@rydmerlin - 26.10.2024 23:52

Why isn’t there a book written on this subject that goes into a deep dive of the solutions?

Ответить

@suraj.mohapatra - 03.11.2024 22:00

I am overwhelmed with the amount of workaround you guys put into. But I understood each. Great work. Kudos.
The code redability is gone with lots of bitwise operations but is good for a challenge.

Ответить