Build 3018 should be available tonight, and it will mark a significant milestone for World Machine…. without even considering what it contains!
Specifically, 3018 will become the basis for the next Release Channel build. Although it’s natural to think of this as “World Machine 3”, we’re moving away from Big Number Releases entirely. In a world where quarterly+ dev channel feature releases and constant improvements are happening to World Machine, Big Number releases simply lose their meaning (unless you like Mozilla-style versions like ‘Firefox 53’…I sure don’t.)
Instead, World Machine will simply be branded as…
World Machine.
Whatever build number it may be.
Of course, some releases are still much more important than others, and it’s helpful to have a way to refer to them. Following in the footprints of Apple, Intel, and others, when a World Machine build contains major new features it will also receive a build name. We’re adopting a whimsical but appropriate version naming system; the build names will be notable mountains or natural features in the Pacific Northwest part of the US, my home.
And so, I introduce to you:
Build 3018 ‘Mailbox Peak’
Mailbox Peak is a local mountain near to Seattle that is beautiful in its own right, but serves especially as notorious training for mountaineers aspiring to larger summits. It’s a fitting name for this build…
Which more than anything, is about getting faster.
Let’s talk about:
- Speed
- UI Improvements
- New Coastal Erosion Device
- Many small changes
Speed
Build 3018’s primary focus is improving build speed for large worlds on modern, many-cored machines.
The subtext here is that I finally updated my 2012-era dev workstation to a modern, high performance machine — one of AMD’s new Threadripper processors. This thing is a beast, with 16/32 cores available, and it was immediately obvious that World Machine was not always putting such parallelism to work. So I set about to change that. You’ll certainly still notice the benefits even on more humble workstations.
Here’s a performance comparison on a selection of example worlds between 3017 and 3018:
Build 3018 is faster across the board, even where 3017 already did quite well. In certainly very common cases, it’s dramatically faster.
As you can see from the above, not only does 3018 bring sometimes-dramatic improvements in speed, but World Machine now scales quite well to high core count machines. Individual easily-scaled devices like Noise show completely linear improvement, while entire world builds aren’t that far behind.
How?
Multicore everything
Virtually all* devices are now multithreaded. Many of these are relatively fast devices like the Combiner that on smaller worlds have a negligible effect. Nonetheless, when building large worlds, speeding up these common devices shaves seconds-to-minutes off the build times.
For completeness, the following devices received multithreading support:
Coastal Erosion, Equalizer, Clamp, Height Curves, Height Select, Slope Select, Select Convexity, Select Aspect, Select Hue, Combiner, Chooser , Channel Combiner, Channel Splitter, Circular Gradient, Constant Height, Constant Color, Height Combiner, Height Splitter, Normal Maker, Splat Converter, Local Placer, Local Scatter, File Input.
*The only remaining 1x devices are essentially just the parameter devices and the Tiled File Input device, which has a separate set of improvements in progress that needed to be coordinated with. That will happen later…
Optimized Implementations
Some optimizations have been made to the Erosion, Advanced Perlin, Snow, and Layout Generator devices. In addition, many internal core operations including mesh creation, grid resizing, etc now use all processor resources.
In particular, the Layout Generator device benefits, now being able to use all threads to build a single shape whereas before each shape was only built by a single thread. In worlds that heavily use layout shapes, this will be a massive speed win.
New Compiler
World Machine has finally been moved to a modern compiler — Visual Studio 2017. This brings with it improved optimizations that on their own improve performance between 0-15%. It should also make writing plugins easier, as the old compiler was difficult to get a hold of these days.
This alone would be enough to merit a new release.
But.. there’s more.
Workview Improvements + Dark-theme
The workview has gained many small tweaks, most notably a new “dark” color scheme.
If you don’t like dark colors, you can switch back to the old default light scheme in program options. Due to the old framework that WM is built on, full dark throughout the UI will have to wait, but all of the OpenGL viewports use the new color scheme.
Also, a new option for wirecolor-by-group has been added. Just like the dark scheme, you can switch back to color-by-type if you’d like, but I think you’ll find that it’s very helpful to be able to color a set of wires according to a purpose.
In addition, various other bits of the workview UI large and small have been tweaked, including:
- Improved grid/snap behavior — this is actually quite helpful as you can finally consistently line devices and groups up properly.
- New “Snap All to Grid” to fix old worlds 😉 and for those who don’t use a grid normally.
- Some improvements to wire routing, highlighting
- Removed device deletion warning thanks to having Undo now
- Pan using arrow keys in Device View
- Default left-mouse drag action is now panning instead of box-select (use shift or ctrl for that)
In general, I spent some time revisiting the Device Workview to try to make its operation more understandable for new users.
New Coastal Erosion Device
The Coastal Erosion device has been improved for 3018. Although operating on essentially the same principles as the previous version (mostly based on modifying height curves appropriately), you’ll find it that it creates more pleasing results as well as being easier to understand and use.
The main goals here were:
- Simplify the parameters, as many of the old ones were not obvious how to best manipulate together
- Create higher quality output, in particular the way that terrain above waterlevel was affected.
- Fix problems that made masks output from the old device hard to work with
Pursuing the first goal, the number of parameters was reduced to the essentials:
While hopefully the first image shows the kinds of results that you can now more easily achieve!
Smaller Things
Finally, a number of things large and small have been tweaked:
- Build Estimator lower bound improved
- River Device goes directly to layout view
- Entering river/layout devices scrolls to the render extents
- Left-side preview says what device is being displayed
- Checkpoint device now auto-names from the input names if nothing else is provided
- Height Curve now lets you specify that it affects only a certain height range. This is incredibly useful! In addition, you can fade the curve effect.
- Height Selector falloff is now specified in meters
- 3D View now uses anisotropic filtering for improved graphics quality, and is smoother and more responsive
- Build Statistics now update faster
- Bugfixes, including “Disable group instead of device” and “3D View doesn’t update when view is frozen”
Build 3018 will go live tonight, hopefully.
The Release Channel version will follow along as quickly as I can make happen — it will contain reworked examples, etc, as it will also become the default choice for new Basic Edition downloads, and that will take some time. And then the WM website in general is getting a refresh to go with it. So there’s lots happening, but Dev Channel people get the action first!
17 replies on “Build 3018.. What’s in a Name?”
This is exciting news!!! Look forward to check it out!
What impact does the compiler change have on the PDK, apart from compiling against VC2017. And any ETA on the PDK?
Very happy to have a more modern compiler! 🙂
I’ll get the PDK up tonight. It should be a transparent upgrade; just upgrade the project files and it will Just Work, depending on if you have to recompile external dependencies…
Great! Will let you know how it goes.
Btw, please give me a headsup if you switch from using ~\AppData\Roaming\World Machine 2.2 Professional\ for the 3.x branch, and any registry paths. We’re using those to detect versions and features, especially when users have more than one WM version installed.
Also, given that you’re dropping version numbers, will there still be an internal build numbering system that developers can rely on for versioning – hopefully, one that is advertised outside of the API for external automation. Right now, we have to rely on MD5 hashes of the 64.exe to get the version right. FYI, the file version and product version advertised by ALL 3.x builds is 3.0.1.0. If you could automate that in your build and have it reflect the actual build numbers, that would go a long way in detecting versions.
Btw, there’s a bug in 3017 and 3018. The “Update macro to toolbar” button is hidden. Makes it impossible to add macros to the menu or toolbar unless you force the button visible via Spy++ or edit the .ini directly.
Hi Dax,
Clicking on any of the checkmarks in the library (including entire folders) will immediately add the macros to the menu/toolbar, the additional step is not needed.
Btw, how difficult would it be for you to recompile the plugin libs and use /MD instead of /MT? With /MD you can then also use /CLR in the plugin dll.
Go go Stephan! This update looks refreshing. On another note I noticed that when copy/pasting nodes in 3017 the relative positions between them get mixed up and becomes a jumbled mess in some cases.
Odd — I haven’t run into this before.. Can you post a set of steps to replicate this problem?
Exciting!!!
I’ve just try your new UI, that’s really cool. It feels like I’m using some other nice UI design software 🙂
And new Multi-thread achieved really makes many device much faster than before, but when I adjust the key-points in Layout Generator device, it becomes very lag, hope you could see that.
And anothor thing, can I type in a number in Reach Elevation datum parameter instaed of drag the buttom? It will be very helpful.
thank you 🙂
Yes exactly my experience.
The updated device view UI is very nice, well done!
I hope you will update the old fashioned and inconsistent dialog boxes too (for example bitmap / heightfield output).
Layout modifications are very slow even on a fast machine.
Entering values instead of using the slider would really be good.
But by looking at the nice and unexpected 3018 improvements I’m looking forward for more great WM development news!
Dear Stephen,
I purchased WM last week while GeoGlyph was on sale. Unfortunately WM won’t use all threads in my system, even with the new update.
I have 2x 18 cores(36 cores, 72 threads) in my machine, but only up to 36 threads are used.
The maximum worker threads count is set to 36 and I can’t raise the value. Via task manager I see it’s only using 50% of the processing power.
Do you have any idea on why this is happening and how to fix this?
I have Hyperthreading enabled in the bios and all other applications work fine and use HT. Tiled builds still don’t use all threads.
My specifications:
Dual Intel Xeon E5-2696V3 (18 cores each)
4x16GB (64GB) DDR4 ECC Registered
Windows 10 Pro Creators Update
Kind regards,
A. Yanik
Hello,
Windows doesn’t transparently present more than 64 cores to an application. I suspect that it has put each physical core into its own processor group. I’ll change the maximum allowed number of threads to count all processor groups. However… I’m honestly not sure of the performance impact of spreading work across multiple processor groups. In my tests it can lead to some trouble, but that may not be the case on your system. In the interim, if you open the file world.ini under %APPDATA%\Roaming\World Machine 2.2 Professional, you can change “System_thread_max” and “Build_thread_max” to 72 manually. The change should persist as long as you don’t save new Preferences. I would be very interested in hearing if this improves performance for your builds!
Hi Stephen,
Thank you for your quick reply.
I’ve modified the .ini file and tested it again. Unfortunately it still didn’t use my second processor even though the build window shows 72 threads. Task manager showed the same CPU usage (between 50-60%).
I really hope you can fix the threading issues. I bought this expensive machine so it could help me with my environment and rendering work. Multithreading across multiple processors works perfectly fine with V-Ray, Houdini or Clarisse. I’m not sure why it is an issue with WM. Could you please fix this in the upcoming builds?
If you need to beta test stuff on my system you are free to do so. Just contact me when needed.
Kind regards,
A. Yanik
I did some googling earlier and found some discussion of this problem, ironically from Intel engineers who had to work around it in their Threading library. I believe I also now have the solution, but unfortunately it will take some testing to hammer out so won’t make it into the Release version of 3018. I will likely take you up on beta-testing a potential fix, as I don’t have a good way to fully simulate/replicate this issue locally. More to come.
I also have a dual Xeon workstation, with a different processor model (E5-2620v3 12 thread) and older Windows version (W7Pro) than Aydin’s machine. Can give you a second data point when working out the threading issues.
Thanks Stephen for that update and I am really glad WM development finally resumed, this is a great software and we are many to appreciate you working on it again. What I would like for WM in 2018 would be:
– make build interruptible in a reliable way (i.e. not having to wait minutes for the build to stop)
– make new stable version (i.e. stop implementing new features and stabilize what is there)
– work on a more user friendly layout generator
Thanks and keep on doing great work on WM!