1. Here you will find official announcements and updates. These announcements are also linked in the Official SotA Discord server.
    We encourage comments from the community! To keep the announcements official, we ask that comment threads be created in the General forums for player input.

                                                 Thanks!

Chris' Release 7 brain dump!

Discussion in 'Announcements' started by Chris, Jul 3, 2014.

Thread Status:
Not open for further replies.
  1. Chris

    Chris Tech Lord Moderator Ambassador SOTA Developer

    Messages:
    2,470
    Likes Received:
    27,551
    Trophy Points:
    190
    Gender:
    Male
    Ok, I’ll throw the usual numbers out so I can dive into the juicy stuff! First off, unique user count was down slightly to 6233 for this release. Not unexpected since we didn’t have any huge new features in this release and now we’re competing for play time with both Wildstar and ESO! My prediction is that those numbers will go up for R8 with the focus being on Combat and PVP.

    The big news is that the mean play session for the more than 21k logins was way up at 2576 seconds a session, or around 43 minutes. If that doesn’t sound like a huge number, remember that this is average which includes all those poor souls who are still on 32 bit and were crashing fairly often (sorry about that and we’ll improve that a bit more for R8). Also, that includes 6208 sessions of 2 minute or less from people who probably tried to login while we were having timeout issues on Friday. Without those bringing down our average, we would have been an hour per session.

    Also, if you’re comparing to R6 stats, remember that this is per session NOT per user. We were at around an hour per user in R6 vs something closer to 3 hours in R7!

    For us, this wasn’t a huge shock. Just adding in more social elements, like being able to have multiple people share lots and a real trading system was really all it took for people to start to work together as a team which helped people bond!

    Ok, now that I got that stuff out of the way, on to my usual babbling!

    So, this was the first release for the new overworld map direction. What people saw was about 2 weeks of research, discovery, and trial and error and then about 3 days of putting it to use. Very much lacking in polish but hopefully enough to give people an idea of the direction. Ignore all the little 3d models that were set on top of the terrain and the trees, as they were all just place holder. The terrain was the near final part and what I’ll discuss for the next few paragraphs!

    The challenge with it was not actually the creation of the map players saw. The challenge was in coming up with a system that met the following requirements:
    • Could be completed in a single release
    • Must be able to scale to a map 1 to 2 orders of magnitude larger
    • Must be generated from data and not hand created or else changes will not be maintainable on a larger scale with frequent updates (so we can do things like easily change seasons or change out large areas based on world events)
    • Must come close to the visual quality of terrain in scenes
    The only thing that was done by hand in the map that people saw was the creation of the height map. That was something that Stephen “Pen of the Avatar” Daniele already had created in Photoshop for past experiments. We also have a way to create the same or better quality height-map from data now if we choose to do so for the main continent but we haven’t decided which we’ll be using.

    The height map just looks like a grey scale image with the high elevation areas being bright and low elevation being dark. We export this from Photoshop in a special “RAW” format so we can store 16 bits of precision instead of 8. We started with just PNG images and 8 bit but when magnified to the terrain, the lack of precision was obvious and resulted in visible terraces in the terrain.

    From this height-map, I created a second mask image for what I considered to be above water level. This was just done by generating a single mask where everything above water level was made pure white or max intensity and everything below water was black. I just did this with some simple Photoshop work. This mask was used so I could amplify just the parts above water to force a more sharp, less natural but more clear coastline. Stephen did a fabulous job but the natural coast line he created just wasn’t working visually.

    These two images were combined in a modified version of a tool called “Terrain Composer”. This allows for the creation of terrain for a large set of definable rules. I used the two images and an intensity to height curve to generate a 4kx4k, or 32 million poly chunk of terrain. For a sanity test, I also used the tool to generate a 40kx40k chunk of tiled terrain or roughly, 3.2 billion polygons to make sure it would scale.. The untextured result of turning the height map into a 32 million polygon terrain mesh looks like this from a distant camera.


    [​IMG]

    The next step was to take that map and come up with a system for creating automatic mappings of terrain types. While there are tools for painting terrain by hand, it is nearly impossible to do smoothly for even small maps, let alone something of this size. Beyond the usual problems of smooth edges, you have to make sure you don’t paint multiple layers down with high intensities and watch for problems in intensities between the multiple “splat maps” used to determine levels of each texture.

    I ended up using a base system that applied sand to everything below a certain level, grass up to a certain elevation level, then rough hills at the next layer, and mountains above that. I added in a bit of logic to increase the effective elevation so cracks and edges show rock more than high elevation but flat areas. To reduce repeating patterns, I then went back and added in some perlin noise in each layer, pulling the next layer down up in light, mottled patches. These are most obvious in the larger grassy areas where it pulled random, non-repeating very light blends of the sand into the grass. It looks a bit odd on close up but worked extremely well to hide the repeating texture issues common in large scale mappings like this. Here is a shot of the sand coming through the grass to hide pattern and simulate grass :

    [​IMG]
    and closer up:
    [​IMG]

    From the hex map, masks for forests, swamps, hill, and desert are created. For the swamp and desert, these masks are used to force down a specific mask blocking the above process and applying sand and swamp textures. The hill texture is used to force more of a brown texture down to reinforce the areas that were already picking up brown from the height map logic pass above. The forest mask was then used to place trees in random scatterings.

    For this version, the 15 or so landmarks were placed by hand. This wasn’t super time consuming so we might continue to do that for the full scale map process. Again, those are just place holders models.

    So one of the unexpected challenges I faced in the above process was choosing map textures that work. We obviously have made tons of maps and Hutch, the environment lead, has a good selection of biome specific stuff picked out. Problem is that while those textures look great in 3rd person mode on a map, they just looked horribly wrong here due to the scale. If you in a scene you expect to be able to see each blade of grass but at this map scale, it just felt weird.

    I’m pretty happy with the textures I ended up with but they look absolutely oddball when used on a scene scale map. The hills were one of the most challenging spots where none of our hill, dirt, rock, or stone textures felt right. Here is the texture I’m referring to in use:

    [​IMG]

    So before I go on, see if you can guess that texture actually is a picture of. Stop looking ahead, guess, it will be fun! :) Ok, now to type some stuff to keep your eyes from wandering to the answer. I found a few stones materials that worked ok but they just didn’t pop. It feels wrong to see the grain of the stone at the scale and they just were all too flat and didn’t have severe enough normals to help fake erosion and detail that isn’t possible even with 32 million polygons. I tried some dirt textures at different resolution but they were also failures. Ok, ready for the answer. The texture you see there is a picture and normal map of bark off an old tree! Yes, really.

    Here is a close up of the image that helps make it a bit more obvious. The part that I loved about it is just how much extra texture and 3d feel it adds due to the intense depth of the cracks in the bark.

    [​IMG]



    Ok, enough talk about the map, lets talk about the epic decorating throw down that happened in Valemark. From what I can tell, the citizens of Valemark decided that they wanted to see if they could break the server. Two competing neighbors were building mazes out crafted cabinets, shelves, and other crafted items. They then decided that just having hundreds of shelves built into a maze was not nearly enough torture for our servers or the frame rates and decided to fill every damn shelf with stuff!

    I encouraged them to keep going to see when things would break. Amazingly, it didn’t break. The frame rate would crawl if you had shadows on but other than that, everything seemed to work. For those who were curious, I think BurningToad checked and found that there were around 5,351 decorations on the lot! Yowsa! I profiled it and found that, thanks to them also adding a ton of lights, the graphics draw calls (how many objects get rendered in a frame) reached 105,000! Most people don’t know what really means but I can assure you that it is insane. Our high target for scenes is around 5000 and peak crowd in worst case Owls Head is around 15000!

    Also, for those who were in Owlshead you probably saw that the frame rate was borderline unplayable with shadows on high on all but the most powerful video cards. Bad times...

    So the good news is that this gave us a good worst case performance situations and inspired a few meetings and some plans. We probably won’t get most stuff implemented until R10, our polish milestone, but I will definitely work to get Owlshead more tolerable for R8 and R9. I think we’re not far off from being able to nail down a “min spec” for a machine as well. Maybe for R10 we’ll make the call on what the minimum machine we’re optimizing for is going to be.

    For those who don’t know, I collect stats on what machines people are playing on. Nothing too invasive, pretty much just CPU, video card, memory, and OS version. Our goal will probably be to make sure that something like 98-99% of people fall into the acceptable performance category once final optimization work is done.

    Now to talk about R8 stuff! Starr’s post will have the rough goals were setting for R8. My goal is simple, make PVP fun. For those in Dev +, I’m going to try to post often over there on what changes are going in and why we are making the various changes. I’ll start with a post on Monday summing up what got done on our first, short week of work on PVP! For those non-Dev+ people, you’ll just have to wait another 3 weeks to see the result yourselves. :)
     
    Samlee, Time Lord, rowan50k and 51 others like this.
  2. Abydos

    Abydos Avatar

    Messages:
    1,827
    Likes Received:
    3,862
    Trophy Points:
    113
    Gender:
    Male
    Location:
    Québec, CAN
    Ur My Best One Man !
     
    Samlee and Time Lord like this.
  3. Akrondar

    Akrondar Avatar

    Messages:
    467
    Likes Received:
    809
    Trophy Points:
    43
    Great! i think we all love your walls of text :cool:
     
  4. Floors

    Floors Avatar

    Messages:
    4,265
    Likes Received:
    6,621
    Trophy Points:
    165
    Informative. I too was impressed that Valemark worked...
     
    Samlee, Time Lord and fstaber like this.
  5. Eriador

    Eriador Avatar

    Messages:
    2,874
    Likes Received:
    5,154
    Trophy Points:
    165
    Location:
    Here!
    Always the best releae thread.
     
  6. Duffrey Blake

    Duffrey Blake Avatar

    Messages:
    720
    Likes Received:
    1,558
    Trophy Points:
    105
    Gender:
    Male
    Location:
    Bravehaven
    Thanks Chris for this very detailed insight of your work. Very appreciated.
     
    Time Lord and abovenyquist like this.
  7. Bolck

    Bolck Avatar

    Messages:
    51
    Likes Received:
    108
    Trophy Points:
    8
    I'll be there for the first time.
     
  8. Gabriel Nightshadow

    Gabriel Nightshadow Avatar

    Messages:
    4,466
    Likes Received:
    9,297
    Trophy Points:
    165
    Gender:
    Male
    Chris,

    Really liked the look of the new overworld map, but the odd thing I noticed in R7 was that when you exited a town, you did not end up in the proper location on the overworld map :( For example, when I exited Valemark, I ended up outside Kingsport on the overworld map o_O Will this be fixed in R8?
     
    Time Lord and Isaiah like this.
  9. Sir Korvash

    Sir Korvash Avatar

    Messages:
    257
    Likes Received:
    699
    Trophy Points:
    40
    Gender:
    Male
    Location:
    Canada
    All I can say is.. wow! Explains why there is a need for an item limit on a plot/house. 105k is insane! I would love to see what the limit was once optimization had been done.



    Personally, i like your "Brain Dumps" the most, as its the tech stuff that i'm really interested in!
     
  10. Dame Lori

    Dame Lori Avatar

    Messages:
    831
    Likes Received:
    3,227
    Trophy Points:
    105
    Gender:
    Female
    Location:
    Bloomington IN
    I really enjoy your post-release messages, Chris! Thanks for the info!
     
  11. Fox Cunning

    Fox Cunning Localization Team

    Messages:
    797
    Likes Received:
    1,645
    Trophy Points:
    93
    Location:
    Wiltshire, England
    Congrats for another successful release, and thanks for this most interesting post.
     
  12. jiirc

    jiirc Avatar

    Messages:
    2,853
    Likes Received:
    2,893
    Trophy Points:
    153
    Chris, thanks again for the tech info. I find it as interesting, if not more interesting, than the other release posts. It's always interesting to see what tec you'll be discussing. Keep it up.

    I'm glad that Riya and her elves were able to provide you with what would surely be worst case scenario. I was shocked that the items that were placed resulted in such a high number of draw calls, that number is, as you said, insane. On the other hand it does speak to the performance capabilities of unity and your hard work. Glad we could help out with providing this information.
     
    Time Lord, fstaber, Khloec and 4 others like this.
  13. Carlin the Druid Archer

    Carlin the Druid Archer Avatar

    Messages:
    833
    Likes Received:
    2,847
    Trophy Points:
    93
    Gender:
    Male
    Location:
    New Britannia
    Brilliant post as always Chris. Looking forward to more in Dev+
     
  14. Lord_Darkmoon

    Lord_Darkmoon Avatar

    Messages:
    4,350
    Likes Received:
    14,680
    Trophy Points:
    153
    A question about the textures of the overland map. Is it possible to have different textures on the same height? So for example different textures of sand and grass? I think that having different textures for grass for example would make the map look more diverse. Brighter grass or darker grass in some spots, maybe some dry grounds etc.

    And will 3D grass and flowers be possible that swing in the wind, at least when you zoom down or do you think that this will look odd?

    Will the map handle more objects like more plants and rocks, clouds etc. Or will this drop the performance too much?
     
    Time Lord likes this.
  15. Floors

    Floors Avatar

    Messages:
    4,265
    Likes Received:
    6,621
    Trophy Points:
    165
    I'm sure the map will evolve, I'm not worried about that. I think it looks cool now. Keep the runic, don't dumb it down !

    I would just like to see towns placed in deserts be deserts when we enter the scene.
     
    Time Lord and Mordecai like this.
  16. Mata

    Mata Avatar

    Messages:
    4,027
    Likes Received:
    9,894
    Trophy Points:
    153
    Gender:
    Female
    Location:
    Ruhrpott - Deutschland
    Thank you for the info!
     
    Time Lord and abovenyquist like this.
  17. Akrondar

    Akrondar Avatar

    Messages:
    467
    Likes Received:
    809
    Trophy Points:
    43
    Maybe a fixed, not too distant, zoom could allow more detail in map without a significant loss in performance. And in my opinion, could be more immersive.
     
    Time Lord likes this.
  18. smack

    smack Avatar

    Messages:
    7,077
    Likes Received:
    15,288
    Trophy Points:
    153
    Thank you Chris for another great tech post. A few followup questions:

    - What factors will come into play for how far back we can pull back the camera on the overland map? Technical / performance limitations? Artistic or gameplay reasons?
    - Will we be able to see only the current island or continent we're on? Or if we're allowed to pull back far enough or pan around far enough, will we see the mainland?
    - There's quite a few folks that would still love to pull back on that camera into a third mode, where it becomes simply a non-interactive paper map (just to view the world). What is the team's current thinking around that?
     
    Time Lord and Lord Trady of Blix like this.
  19. Duke Olahorand

    Duke Olahorand Localization Team

    Messages:
    1,287
    Likes Received:
    2,465
    Trophy Points:
    125
    Gender:
    Male
    Location:
    North Germany
    These server connection timeouts at 94% loading after patch 112 for some people in Valemark - where they triggered by client, by server or both?
    *Salute*
    Olahorand
     
    Time Lord likes this.
  20. Isaiah

    Isaiah Avatar

    Messages:
    6,887
    Likes Received:
    8,359
    Trophy Points:
    165
    Gender:
    Male
    Did you submit this as a bug report :mad: ?
     
    Time Lord likes this.
Thread Status:
Not open for further replies.