ParaLily Update #4: Improving Level Design with Player Analytics and Heatmaps
In our last update we showed a bunch of Player Analytics that we collected from MAGFest including playtime, death variations, levels beaten, etc. Now we’d like to share how we collected, summarized, and utilized all that data to help improve ParaLily.
As a two man team we spend a lot of time not only designing and creating ParaLily, but also playing testing, ParaLily. Unfortunately all that testing really skews our perception of difficulty within the game. In order to combat this we have to let people test our game and see for ourselves where things are too hard or confusing. Fortunately conventions are great place to get feedback! We can watch players firsthand, and see where they get stuck or lost. Even better, we can make sure we don’t miss a beat, by keeping track of our players events in an audit log file.
Logging
The backbone of our analytic system relies heavily on Logging. Specifically Audit Logging, which is a method of recording chronological events. For an everyday example someone might log when they ate an apple. It’s important to note that we don’t care how many apples they ate, we just log each one as a separate entry like so:
ate apple ate apple ate apple
Now if we want to know where each apple was eaten, or what kind of specific apple it was, then we just add that to our log entry:
ate apple fuji kitchen ate apple gala office ate apple fuji bedroom
Looking at the logs we can see that three apples were eaten. Two of which were fuji, and were eaten at home. So from this bit of data we can assume that at home there are fuji apples and at work there are gala apples.
In ParaLily we apply this same concept where each line in our log file is simply an event that the Player executed. It ends up looking something like this:
04d17h27m00s Game Start 0f4fbaf6-9e9a-4740-a01a-86369f74419e 04d17h27m24s 1-1 Level Start 04d17h27m48s 1-1 r1 81.6,7.7 Glyph 04d17h27m54s 1-1 r1 131.4,1.3 Glyph 04d17h28m00s 1-1 Level Win 35.92s 04d17h28m01s 1-2 Level Start 04d17h28m08s 1-2 r1 -4.8,-5.0 SXB 04d17h28m08s 1-2 r1 -0.3,-24.3 Death Fall 04d17h28m09s 1-2 Level Lose 7.45s 04d17h28m10s 1-2 Level Start 04d17h28m14s 1-2 r1 -25.8,-1.9 SB 04d17h28m31s 1-2 r2 16.6,-3.6 SF 04d17h28m35s 1-2 r1 16.9,-3.2 Glyph 04d17h28m37s 1-2 r1 16.9,-0.1 SB 04d17h28m42s 1-2 r2 33.9,0.5 SF 04d17h28m47s 1-2 r1 63.8,-2.1 SB 04d17h28m53s 1-2 r2 87.9,0.3 SF 04d17h28m54s 1-2 r1 89.9,-3.0 Death Thorn 04d17h28m54s 1-2 Level Lose 44.25s 04d17h28m56s 1-2 Level Start 04d17h29m02s 1-2 r1 -19.2,-0.3 SB 04d17h29m11s 1-2 r2 35.1,-1.1 SF 04d17h29m17s 1-2 r1 65.8,0.0 SB 04d17h29m22s 1-2 r2 71.6,-5.8 SF 04d17h29m23s 1-2 r1 71.6,-5.3 Glyph 04d17h29m25s 1-2 r1 75.9,-3.4 SB 04d17h29m27s 1-2 r2 82.4,-2.6 SF 04d17h29m31s 1-2 r1 93.6,-4.5 SB 04d17h29m32s 1-2 r2 98.2,-20.5 SF 04d17h29m33s 1-2 r1 98.9,-24.1 Death Fall 04d17h29m33s 1-2 Level Lose 37.52s 04d17h29m35s 1-2 Level Start 04d17h29m39s 1-2 r1 -23.6,-1.1 SB 04d17h29m49s 1-2 r2 41.3,-0.6 SF 04d17h30m00s 1-2 r1 101.8,-1.3 SB 04d17h30m06s 1-2 r2 130.2,1.1 SF 04d17h30m10s 1-2 Level Win 35.15s 04d17h30m11s 1-3 Level Start 04d17h30m19s 1-3 r1 19.8,1.8 Glyph 04d17h30m22s 1-3 r1 31.3,0.7 SB 04d17h30m31s 1-3 r2 61.2,-0.3 SB 04d17h30m42s 1-3 r3 90.8,-6.7 Glyph 04d17h30m43s 1-3 r3 94.6,-9.6 SF 04d17h30m47s 1-3 r2 110.6,-2.5 Glyph 04d17h30m52s 1-3 r2 146.5,-1.3 SB 04d17h30m54s 1-3 r3 151.2,-10.9 SF 04d17h30m55s 1-3 r2 152.6,-24.2 Death Fall 04d17h30m55s 1-3 Level Lose 43.77s 04d17h30m57s 1-3 Level Start 04d17h31m06s 1-3 r1 31.4,0.4 SB 04d17h31m13s 1-3 r2 59.8,0.0 SB 04d17h31m21s 1-3 r3 98.0,-12.4 SXF 04d17h31m21s 1-3 r3 98.1,-12.9 Death LavaFlow 04d17h31m21s 1-3 Level Lose 24.46s 04d17h31m23s 1-3 Level Start 04d17h31m32s 1-3 r1 31.3,-0.1 SB 04d17h31m41s 1-3 r2 59.4,-0.5 SB 04d17h31m49s 1-3 r3 95.2,-7.3 SF 04d17h32m02s 1-3 r2 154.0,-24.2 Death Fall 04d17h32m03s 1-3 Level Lose 40.07s 04d17h32m04s 1-3 Level Start 04d17h32m14s 1-3 r1 31.8,0.0 SB 04d17h32m19s 1-3 r2 59.1,-2.0 SF 04d17h32m20s 1-3 r1 59.2,-2.3 SB 04d17h32m22s 1-3 r2 64.4,-0.2 SF 04d17h32m24s 1-3 r1 67.0,-24.3 Death Fall 04d17h32m24s 1-3 Level Lose 20.14s 04d17h32m26s 1-3 Level Start 04d17h32m35s 1-3 r1 30.4,0.4 SB 04d17h32m42s 1-3 r2 60.6,-0.5 SB 04d17h32m49s 1-3 r3 91.5,-7.1 SF 04d17h33m00s 1-3 r2 145.0,-1.8 SF 04d17h33m01s 1-3 r1 146.7,-1.0 SB 04d17h33m02s 1-3 r2 148.8,-1.7 SF 04d17h33m05s 1-3 Level Win 39.18s 04d17h33m06s 1-4 Level Start 04d17h33m13s 1-4 r1 14.4,2.4 Glyph 04d17h33m17s 1-4 r1 34.2,5.0 SB 04d17h33m22s 1-4 r2 49.2,1.6 SB 04d17h33m24s 1-4 r3 49.2,2.0 SF 04d17h33m25s 1-4 r2 49.2,1.6 SF 04d17h33m26s 1-4 r1 49.2,0.4 SXB 04d17h33m31s 1-4 r1 71.5,-24.2 Death Fall 04d17h33m32s 1-4 Level Lose 25.30s 04d17h33m33s 1-4 Level Start 04d17h33m43s 1-4 r1 34.8,4.9 SB 04d17h33m46s 1-4 r2 48.3,1.8 SF 04d17h33m49s 1-4 r1 49.2,-1.2 SXB 04d17h33m50s 1-4 r1 49.2,-1.6 SB 04d17h33m51s 1-4 r2 49.2,-1.6 Glyph 04d17h33m54s 1-4 r2 51.7,-4.4 SF 04d17h33m56s 1-4 r1 55.9,-1.3 SB 04d17h34m00s 1-4 r2 51.5,3.7 SB 04d17h34m24s 1-4 r3 80.2,-5.8 SF 04d17h34m25s 1-4 r2 80.2,-7.0 SF 04d17h34m26s 1-4 r1 80.2,-12.5 SXB 04d17h34m26s 1-4 r1 80.2,-24.2 Death Fall 04d17h34m27s 1-4 Level Lose 53.39s 04d17h34m28s 1-4 Level Start 04d17h34m36s 1-4 r1 29.0,6.0 SB 04d17h34m40s 1-4 r2 53.1,3.5 SB 04d17h34m46s 1-4 r3 79.6,-9.2 SF 04d17h34m49s 1-4 r2 87.7,3.2 SB 04d17h34m51s 1-4 r3 84.3,3.3 Glyph 04d17h34m52s 1-4 r3 88.0,2.6 SF 04d17h34m59s 1-4 r2 134.0,2.6 SF 04d17h35m02s 1-4 Level Win 33.86s 04d17h35m03s 1-5 Level Start 04d17h35m11s 1-5 r1 -78.5,-1.2 Death Bunny 04d17h35m11s 1-5 Level Lose 7.72s 04d17h35m13s 1-5 Level Start 04d17h35m30s 1-5 r1 -60.4,2.3 SB 04d17h35m35s 1-5 r2 -43.6,2.1 SB 04d17h35m36s 1-5 r3 -38.5,-12.0 Death LavaFlow 04d17h35m37s 1-5 Level Lose 24.10s 04d17h35m38s 1-5 Level Start 04d17h35m48s 1-5 r1 -59.9,2.1 SB 04d17h35m52s 1-5 r2 -43.3,2.1 SF 04d17h35m53s 1-5 r1 -41.8,0.5 SB 04d17h35m55s 1-5 r2 -45.5,1.6 SF 04d17h35m56s 1-5 r1 -45.5,1.6 Glyph 04d17h35m56s 1-5 r1 -45.4,1.1 SB 04d17h35m57s 1-5 r2 -44.0,-5.8 SB 04d17h35m58s 1-5 r3 -42.8,-12.1 Death LavaFlow 04d17h35m58s 1-5 Level Lose 20.09s 04d17h36m00s 1-5 Level Start 04d17h36m05s 1-5 r1 -86.8,-3.0 Death Bunny 04d17h36m05s 1-5 Level Lose 5.64s 04d17h36m07s 1-5 Level Start 04d17h36m16s 1-5 r1 -60.7,2.1 SB 04d17h36m19s 1-5 r2 -43.9,2.1 SB 04d17h36m20s 1-5 r3 -42.0,1.4 SF 04d17h36m22s 1-5 r2 -34.1,3.5 SB 04d17h36m23s 1-5 r3 -31.2,3.6 SF 04d17h36m24s 1-5 r2 -29.4,0.6 SF 04d17h36m27s 1-5 r1 -14.2,-1.6 Glyph 04d17h36m28s 1-5 r1 -12.7,-1.6 SB 04d17h36m32s 1-5 r2 1.5,2.3 SF 04d17h36m33s 1-5 r1 6.0,-0.7 SB 04d17h36m34s 1-5 r2 7.1,-3.6 SB 04d17h36m35s 1-5 r3 7.5,-5.5 Death SkullSpider 04d17h36m35s 1-5 Level Lose 28.16s 04d17h36m37s 1-5 Level Start 04d17h36m46s 1-5 r1 -60.0,2.1 SB 04d17h36m49s 1-5 r2 -42.8,1.7 SB 04d17h36m50s 1-5 r3 -40.7,-1.5 SXF 04d17h36m50s 1-5 r3 -37.7,-12.1 Death LavaFlow 04d17h36m50s 1-5 Level Lose 13.77s 04d17h36m52s 1-5 Level Start 04d17h37m01s 1-5 r1 -59.1,1.7 SB 04d17h37m05s 1-5 r2 -34.9,3.4 SF 04d17h37m08s 1-5 r1 -14.7,-1.1 SB 04d17h37m12s 1-5 r2 1.1,2.3 SB 04d17h37m14s 1-5 r3 6.6,-5.6 Death SkullSpider 04d17h37m14s 1-5 Level Lose 22.10s 04d17h37m15s 1-5 Level Start 04d17h37m24s 1-5 r1 -59.8,1.7 SB 04d17h37m28s 1-5 r2 -36.3,2.7 SXF 04d17h37m29s 1-5 r2 -31.8,-1.3 SF 04d17h37m32s 1-5 r1 -14.0,-1.4 SB 04d17h37m36s 1-5 r2 8.1,-0.6 SB 04d17h37m37s 1-5 r3 9.7,-6.0 Death SkullSpider 04d17h37m37s 1-5 Level Lose 22.04s 04d17h37m39s 1-5 Level Start 04d17h37m48s 1-5 r1 -58.7,1.4 SB 04d17h37m52s 1-5 r2 -36.3,2.6 SXF 04d17h37m52s 1-5 r2 -32.8,-0.6 SF 04d17h37m56s 1-5 r1 -14.0,-1.1 SB 04d17h37m59s 1-5 r2 3.0,4.0 SF 04d17h38m00s 1-5 r1 4.1,4.2 SB 04d17h38m01s 1-5 r2 7.1,1.5 SB 04d17h38m02s 1-5 r3 9.7,-5.8 Death SkullSpider 04d17h38m02s 1-5 Level Lose 23.44s 04d17h38m04s 1-5 Level Start 04d17h38m13s 1-5 r1 -59.6,1.9 SB 04d17h38m17s 1-5 r2 -34.6,3.4 SF 04d17h38m20s 1-5 r1 -14.1,-1.0 SB 04d17h38m23s 1-5 r2 0.3,3.5 SB 04d17h38m24s 1-5 r3 1.8,4.0 SF 04d17h38m25s 1-5 r2 3.1,3.6 SB 04d17h38m26s 1-5 r3 7.3,-5.8 Death SkullSpider 04d17h38m27s 1-5 Level Lose 22.79s 04d17h38m28s 1-5 Level Start 04d17h38m37s 1-5 r1 -61.9,1.5 SB 04d17h38m41s 1-5 r2 -35.9,3.2 SXF 04d17h38m42s 1-5 r2 -30.9,-1.8 SF 04d17h38m45s 1-5 r1 -14.0,-1.1 SB 04d17h38m49s 1-5 r2 2.2,4.1 SB 04d17h38m53s 1-5 r3 17.6,-0.9 SF 04d17h38m55s 1-5 r2 22.8,0.0 SF 04d17h38m56s 1-5 r1 20.9,-24.3 Death Fall 04d17h38m57s 1-5 Level Lose 28.61s 04d17h38m58s 1-5 Level Start 04d17h39m07s 1-5 r1 -58.6,1.4 SB 04d17h39m12s 1-5 r2 -33.3,2.0 SF 04d17h39m15s 1-5 r1 -15.5,-1.4 SB 04d17h39m18s 1-5 r2 0.5,-0.3 SF 04d17h39m19s 1-5 r1 2.8,2.3 SB 04d17h39m21s 1-5 r2 6.0,-3.4 SB 04d17h39m22s 1-5 r3 4.8,-6.2 Death SkullSpider 04d17h39m22s 1-5 Level Lose 24.21s 04d17h39m24s 1-5 Level Start 04d17h39m33s 1-5 r1 -59.2,1.8 SB 04d17h39m37s 1-5 r2 -33.7,1.6 SF 04d17h39m41s 1-5 r1 -13.5,-1.1 SB 04d17h39m44s 1-5 r2 0.0,2.9 SF 04d17h39m45s 1-5 r1 1.0,3.8 SB 04d17h39m46s 1-5 r2 3.8,-5.2 SB 04d17h39m50s 1-5 r3 16.0,-1.6 SF 04d17h39m52s 1-5 r2 20.9,3.1 SB 04d17h39m55s 1-5 r3 30.1,2.9 SF 04d17h39m57s 1-5 r2 34.5,3.7 SB 04d17h39m59s 1-5 r3 40.3,3.7 SF 04d17h40m04s 1-5 r2 38.1,8.3 SB 04d17h40m06s 1-5 r3 38.2,-5.4 SF 04d17h40m07s 1-5 r2 38.8,-6.2 SB 04d17h40m09s 1-5 r3 38.5,-7.5 SF 04d17h40m10s 1-5 r2 37.8,-6.3 Glyph 04d17h40m11s 1-5 r2 37.7,-3.3 SF 04d17h40m12s 1-5 r1 37.8,-0.8 SB 04d17h40m16s 1-5 r2 56.0,3.2 SB 04d17h40m17s 1-5 r3 59.1,4.6 SF 04d17h40m19s 1-5 r2 62.1,1.1 SF 04d17h40m25s 1-5 r1 99.8,-24.3 Death Fall 04d17h40m25s 1-5 Level Lose 60.89s 04d17h40m26s 1-5 Level Start 04d17h40m35s 1-5 r1 -59.3,1.8 SB 04d17h40m40s 1-5 r2 -35.4,3.1 SF 04d17h40m43s 1-5 r1 -15.4,-1.2 SB 04d17h40m46s 1-5 r2 0.2,1.6 SB 04d17h40m49s 1-5 r3 4.0,-5.8 Death SkullSpider 04d17h40m49s 1-5 Level Lose 22.35s 04d17h40m50s 1-5 Level Start 04d17h40m59s 1-5 r1 -59.8,2.1 SB 04d17h41m03s 1-5 r2 -35.5,3.1 SF 04d17h41m07s 1-5 r1 -14.5,-1.0 SB 04d17h41m10s 1-5 r2 -2.1,2.9 SF 04d17h41m11s 1-5 r1 3.0,-2.0 SB 04d17h41m12s 1-5 r2 3.7,-14.2 SB 04d17h41m13s 1-5 r3 3.7,-14.2 Death LavaFlow 04d17h41m13s 1-5 Level Lose 23.19s 04d17h41m15s 1-5 Level Start 04d17h41m24s 1-5 r1 -59.4,1.8 SB 04d17h41m28s 1-5 r2 -33.3,-0.6 SF 04d17h41m32s 1-5 r1 -13.6,-1.2 SB 04d17h41m35s 1-5 r2 0.7,2.3 SB 04d17h41m40s 1-5 r3 18.0,-3.3 SF 04d17h41m42s 1-5 r2 20.5,3.4 SB 04d17h41m44s 1-5 r3 27.0,3.6 SF 04d17h41m50s 1-5 r2 56.4,1.8 SF 04d17h42m00s 1-5 Level Win 45.15s 04d17h42m02s 1-6 Level Start 04d17h42m14s 1-6 r1 34.6,-3.5 Death Bunny 04d17h42m14s 1-6 Level Lose 11.95s 04d17h42m16s 1-6 Level Start 04d17h42m29s 1-6 r1 49.8,-2.0 SB 04d17h42m31s 1-6 r2 46.9,1.6 Death Creep 04d17h42m31s 1-6 Level Lose 15.42s 04d17h42m33s 1-6 Level Start 04d17h42m44s 1-6 r1 35.5,2.8 SB 04d17h42m51s 1-6 r2 76.2,0.8 SB 04d17h42m55s 1-6 r3 98.4,1.3 SF 04d17h42m59s 1-6 r2 120.3,-8.4 SB 04d17h43m00s 1-6 r3 120.7,-10.0 SXF 04d17h43m00s 1-6 r3 120.9,-12.1 Death LavaFlow 04d17h43m01s 1-6 Level Lose 27.53s 04d17h43m02s 1-6 Level Start 04d17h43m14s 1-6 r1 35.5,2.8 SB 04d17h43m21s 1-6 r2 76.8,1.0 SB 04d17h43m25s 1-6 r3 96.4,0.9 SF 04d17h43m31s 1-6 r2 137.1,0.5 SF 04d17h43m33s 1-6 r1 152.8,0.5 SB 04d17h43m35s 1-6 r2 160.0,0.4 SB 04d17h43m38s 1-6 r3 175.8,0.2 SF 04d17h43m42s 1-6 r2 197.3,1.1 SB 04d17h43m44s 1-6 r3 211.2,-2.7 SF 04d17h43m46s 1-6 r2 220.6,1.2 SB 04d17h43m52s 1-6 r3 250.8,2.7 Death LavaPool 04d17h43m52s 1-6 Level Lose 49.95s 04d17h43m54s 1-6 Level Start 04d17h44m06s 1-6 r1 36.4,2.8 SB 04d17h44m12s 1-6 r2 76.2,1.1 SB 04d17h44m16s 1-6 r3 96.5,0.9 SF 04d17h44m22s 1-6 r2 135.2,2.5 SF 04d17h44m25s 1-6 r1 152.9,0.7 SB 04d17h44m27s 1-6 r2 159.1,1.0 SB 04d17h44m30s 1-6 r3 175.5,1.4 SF 04d17h44m33s 1-6 r2 197.8,1.1 SB 04d17h44m36s 1-6 r3 211.5,-2.5 SF 04d17h44m38s 1-6 r2 221.8,2.8 SB 04d17h44m41s 1-6 r3 237.5,-3.0 Death SkullSpider 04d17h44m42s 1-6 Level Lose 47.46s 04d17h44m43s 1-6 Level Start 04d17h44m55s 1-6 r1 36.8,2.8 SB 04d17h45m02s 1-6 r2 76.6,1.0 SB 04d17h45m05s 1-6 r3 96.6,1.5 SF 04d17h45m12s 1-6 r2 136.3,2.6 SF 04d17h45m15s 1-6 r1 153.6,0.7 SB 04d17h45m17s 1-6 r2 161.1,1.3 SB 04d17h45m20s 1-6 r3 175.3,1.6 SF 04d17h45m23s 1-6 r2 197.1,1.0 SB 04d17h45m26s 1-6 r3 212.5,-3.0 SF 04d17h45m27s 1-6 r2 218.5,0.9 SB 04d17h45m32s 1-6 r3 231.1,-2.1 Death Creep 04d17h45m32s 1-6 Level Lose 48.82s 04d17h45m34s 1-6 Level Start 04d17h45m46s 1-6 r1 37.1,2.7 SB 04d17h45m52s 1-6 r2 76.3,1.0 SB 04d17h45m55s 1-6 r3 98.1,1.7 SF 04d17h46m01s 1-6 r2 135.6,2.6 SF 04d17h46m04s 1-6 r1 151.5,0.1 SB 04d17h46m06s 1-6 r2 160.4,1.3 SB 04d17h46m08s 1-6 r3 172.9,2.8 SF 04d17h46m12s 1-6 r2 197.5,0.9 SB 04d17h46m15s 1-6 r3 210.6,-2.4 SF 04d17h46m17s 1-6 r2 221.9,2.0 SB 04d17h46m25s 1-6 r3 254.8,5.2 Glyph 04d17h46m31s 1-6 r3 291.6,-5.6 SF 04d17h46m35s 1-6 r2 316.3,-2.3 SF 04d17h46m38s 1-6 Level Win 63.54s 04d17h46m38s Game End 1177.96s 0f4fbaf6-9e9a-4740-a01a-86369f74419e
So what’s going on in all this mess? Just like in the apple scenario a space is our delimiter for each column. So our columns are roughly:
DATE/TIME LEVEL ROW PLAYER_POSITION EVENT
Date and time are great to have because it let’s us see just how long it took players to go from one place to another. Then in ParaLily we not only have levels like world 1 level 1, but also Universes or Rows as we call them, which are r1, r2, and r3. Logging the Players exact position gives us a lot of opportunities like generating heatmaps. Lastly, we log the event itself: Death, Shift Forward, Shift Back, or Glyph Collected.
We also have special scenarios like the game starting or ending, a level starting or ending. Timestamps for key things like when the game ends, in this case they beat it after 1177.96s. A Unique Identifier is generated per play-through, and helps us find specific logs if something crazy happens in game.
Then when the next player comes along we create a new log file. So each log file itself is a single play-through. Our folder by the end of a convention:
For developers looking to setup their own loggers it’s pretty straightforward. My biggest tip would be to make sure you are not re-opening the log file every time you write to it. Here’s the code we used for our Audit Logger:
using System;
using System.IO;
using System.Text;
using UnityEngine;
public static class Logger
{
private static readonly string[] Row = new string[] { " r1 ", " r2 ", " r3 " };
private static readonly string LogDir = Path.Combine(Application.persistentDataPath, "Logs/");
private static readonly string FileName = "_ParaLily_audit.log";
private static readonly string FileDateTimeFormat = "yyyy-MM-dd'T'HH'h'mm'm'ss's'";
private static readonly string LogDateTimeFormat = "dd'd'HH'h'mm'm'ss's'";
public static Guid Uid { get; set; }
private static string auditLogPath;
private static float gameStart;
private static float playStart;
private static StreamWriter logWriter;
static Logger()
{
SetNewGame();
if (!Directory.Exists(LogDir))
{
Directory.CreateDirectory(LogDir);
}
}
public static void AuditGame()
{
SetNewGame();
StringBuilder sb = new StringBuilder();
sb.Append(DateTime.Now.ToString(LogDateTimeFormat)).Append(" Game Start ").Append(Uid.ToString());
WriteLog(sb.ToString());
}
public static void AuditGame(string text)
{
StringBuilder sb = new StringBuilder();
sb.Append(DateTime.Now.ToString(LogDateTimeFormat)).Append(" Game ").Append(text).Append(" ")
.Append(GameDurationInSecondsToString()).Append(" ").Append(Uid.ToString());
WriteLog(sb.ToString());
}
public static void AuditLevelStart()
{
StringBuilder sb = new StringBuilder();
playStart = Time.time;
sb.Append(DateTime.Now.ToString(LogDateTimeFormat)).Append(" ").Append(GameManager.instance.CurrentSceneName).Append(" Level Start");
WriteLog(sb.ToString());
}
public static void AuditLevel(string text)
{
StringBuilder sb = new StringBuilder();
sb.Append(DateTime.Now.ToString(LogDateTimeFormat)).Append(" ").Append(GameManager.instance.CurrentSceneName).Append(" Level ").Append(text).Append(" ").Append(PlayDurationInSecondsToString());
WriteLog(sb.ToString());
}
public static void AuditFull(string text)
{
StringBuilder sb = new StringBuilder();
sb.Append(DateTime.Now.ToString(LogDateTimeFormat)).Append(" ").Append(GameManager.instance.CurrentSceneName).Append(Row[RowManager.instance.CurrentParallel])
.Append(PlayerController.instance.transform.position.ToStringXY()).Append(" ").Append(text);
WriteLog(sb.ToString());
}
private static void WriteLog(string text)
{
logWriter.WriteLine(text);
logWriter.Flush();
}
public static float GameDurationInSeconds()
{
return Time.time - gameStart;
}
public static float PlayDurationInSeconds()
{
return Time.time - playStart;
}
private static string GameDurationInSecondsToString()
{
return GameDurationInSeconds().ToString("0.00s");
}
private static string PlayDurationInSecondsToString()
{
return PlayDurationInSeconds().ToString("0.00s");
}
private static void SetNewGame()
{
Uid = Guid.NewGuid();
gameStart = Time.time;
auditLogPath = Path.Combine(LogDir, DateTime.Now.ToString(FileDateTimeFormat) + FileName);
Dispose();
logWriter = new StreamWriter(auditLogPath);
logWriter.AutoFlush = false;
}
public static void Dispose()
{
logWriter?.Dispose();
}
}
Summarizing with Shell Scripting
So what do we do with all of this data? There’s hundreds of lines per file and hundreds of files. It would take forever to sift through and tally all this up by hand. Fear not! Scripting is here to save us from all that misery. To get all the important bits of data we want, we created a shell script to do all the heavy lifting, and generate a clean summarized output of all that player data. After it runs it leaves us with a folder with all of these files:
The files each look similar to this:
Game Start 109 Game End 7 Game Timeout 59 Game Reset 27 Level Start 1707 Level Win 191 Level Lose 1427 SB 2893 SF 1996 SXB 322 SXF 360 Glyph 576 Death 1427 Death Thorn 373 Death Fall 790 Death SkullSpider 57 Death LavaPool 24 Death LavaFlow 116 Death Creep 12 Death Bunny 16 Death Quicksand 39
*Data collected from Cecil Con
And here’s the Shell Script we developed to do all the summarizing:
#!/bin/bash
function date_time_format {
date "+%Y-%m-%d"
}
SUMMARY_DIR="summary_$(date_time_format)"
LEVELS=( "1-1" "1-2" "1-3" "1-4" "1-5" "1-6" "1-7" "1-8" )
ROWS=( "r1" "r2" "r3" )
DAYS=( "14d" ) # Days the event ran
HOURS=( "10h" "11h" "12h" "13h" "14h" "15h" "16h" "17h" )
#HOURS=( "00h" "01h" "02h" "03h" "04h" "05h" "06h" "07h" "08h" "09h" "10h" "11h" "12h" "13h" "14h" "15h" "16h" "17h" "18h" "19h" "20h" "21h" "22h" "23h" )
KEYWORDS_BY_LEVEL_BY_ROW=( "SB" "SF" "SXB" "SXF" "Glyph" "Death" )
KEYWORDS_BY_LEVEL=( "Level Start" "Level Win" "Level Lose" "Level Reset" "Level Timeout" "SB" "SF" "SXB" "SXF" "Glyph" "Death" "Death Thorn" "Death Fall" "Death SkullSpider" "Death LavaPool" "Death LavaFlow" "Death Creep" "Death Bunny" "Death Quicksand" )
KEYWORDS_BY_ROW=( "SB" "SF" "SXB" "SXF" "Glyph" "Death" )
KEYWORDS_OVERALL=( "Game Start" "Game End" "Game Timeout" "Game Reset" "Level Start" "Level Win" "Level Lose" "SB" "SF" "SXB" "SXF" "Glyph" "Death" "Death Thorn" "Death Fall" "Death SkullSpider" "Death LavaPool" "Death LavaFlow" "Death Creep" "Death Bunny" "Death Quicksand" )
KEYWORDS_BY_COORDINATE=( "SB" "SF" "SXB" "SXF" "Glyph" "Death" )
KEYWORDS_BY_HOUR=( "Game Start" "Game End" "Game Timeout" "Game Reset" "Level Start" "Level Win" "Level Lose" "SB" "SF" "SXB" "SXF" "Glyph" "Death" "Death Thorn" "Death Fall" "Death SkullSpider" "Death LavaPool" "Death LavaFlow" "Death Creep" "Death Bunny" "Death Quicksand" )
cd '.\Logs'
mkdir $SUMMARY_DIR
echo "Building Analytic Summary to $SUMMARY_DIR..."
#--Totals by Level by Row Summary--
TOTALS_BY_LEVEL_BY_ROW_FILE="$SUMMARY_DIR\\totals_by_level_by_row.txt"
echo "Generating File $TOTALS_BY_LEVEL_BY_ROW_FILE..."
for (( i=0; i<${#LEVELS[@]}; i++ ))
do
for (( k=0; k<${#ROWS[@]}; k++ ))
do
for (( j=0; j<${#KEYWORDS_BY_LEVEL_BY_ROW[@]}; j++ ))
do
COUNT[$j]=$(grep "${LEVELS[$i]}.*${ROWS[$k]}.*${KEYWORDS_BY_LEVEL_BY_ROW[$j]}" *audit* | wc -l )
echo "${LEVELS[$i]} ${ROWS[$k]} ${KEYWORDS_BY_LEVEL_BY_ROW[$j]} ${COUNT[$j]}" | tee -a $TOTALS_BY_LEVEL_BY_ROW_FILE
done
done
done
#--Totals by Row Summary--
TOTALS_BY_ROW_FILE="$SUMMARY_DIR\\totals_by_row.txt"
echo "Generating File $TOTALS_BY_ROW_FILE..."
for (( i=0; i<${#ROWS[@]}; i++ ))
do
for (( k=0; k<${#KEYWORDS_BY_ROW[@]}; k++ ))
do
COUNT[$k]=$(grep "${ROWS[$i]}.*${KEYWORDS_BY_ROW[$k]}" *audit* | wc -l )
echo "${ROWS[$i]} ${KEYWORDS_BY_ROW[$k]} ${COUNT[$k]}" | tee -a $TOTALS_BY_ROW_FILE
done
done
#--Totals by Level Summary--
TOTALS_BY_LEVEL_FILE="$SUMMARY_DIR\\totals_by_level.txt"
echo "Generating File $TOTALS_BY_LEVEL_FILE..."
for (( i=0; i<${#LEVELS[@]}; i++ ))
do
for (( k=0; k<${#KEYWORDS_BY_LEVEL[@]}; k++ ))
do
COUNT[$k]=$(grep "${LEVELS[$i]}.*${KEYWORDS_BY_LEVEL[$k]}" *audit* | wc -l )
echo "${LEVELS[$i]} ${KEYWORDS_BY_LEVEL[$k]} ${COUNT[$k]}" | tee -a $TOTALS_BY_LEVEL_FILE
done
done
#--Totals Overall Summary--
TOTALS_FILE="$SUMMARY_DIR\\totals_overall.txt"
echo "Generating File $TOTALS_FILE..."
for (( i=0; i<${#KEYWORDS_OVERALL[@]}; i++ ))
do
COUNT[$i]=$(grep "${KEYWORDS_OVERALL[$i]}" *audit* | wc -l )
echo "${KEYWORDS_OVERALL[$i]} ${COUNT[$i]}" | tee -a $TOTALS_FILE
done
#--Coordinate Summary--
echo "Generating Coordinate Files..."
for (( i=0; i<${#LEVELS[@]}; i++ ))
do
coordinate_file="$SUMMARY_DIR\\${LEVELS[$i]}_coordinates.txt"
for (( k=0; k<${#KEYWORDS_BY_COORDINATE[@]}; k++ ))
do
for (( j=0; j<${#ROWS[@]}; j++ ))
do
grep "${LEVELS[$i]} ${ROWS[j]}.*${KEYWORDS_BY_COORDINATE[$k]}" *audit* | awk '{print $5,$3,$4;}' >> "$coordinate_file"
done
done
echo "Generated $coordinate_file."
done
#--Hourly Totals Summary--
HOURLY_FILE="$SUMMARY_DIR\\totals_by_day_by_hour.txt"
echo "Generating File $HOURLY_FILE..."
for (( i=0; i<${#DAYS[@]}; i++))
do
for (( k=0; k<${#HOURS[@]}; k++ ))
do
for (( j=0; j<${#KEYWORDS_BY_HOUR[@]}; j++ ))
do
count=$(grep "${DAYS[$i]}${HOURS[$k]}.*${KEYWORDS_BY_HOUR[$j]}" *audit* | wc -l)
echo "${DAYS[$i]}${HOURS[$k]} ${KEYWORDS_BY_HOUR[$j]} $count" | tee -a "$HOURLY_FILE"
done
# Game time by Day by Hour
hourly_game_time=$(grep "${DAYS[$i]}${HOURS[$k]}.*Game End\|${DAYS[$i]}${HOURS[$k]}.*Game Timeout\|${DAYS[$i]}${HOURS[$k]}.*Game Reset" *audit* | sed 's/s//g' | awk '{sum += $4} END {print sum}')
echo "${DAYS[$i]}${HOURS[$k]} Playtime $hourly_game_time" | tee -a "$HOURLY_FILE"
done
done
#--Playtime Summary--
PLAYTIME_FILE="$SUMMARY_DIR\\playtime.txt"
echo "Generating File $PLAYTIME_FILE..."
for (( i=0; i<${#LEVELS[@]}; i++ ))
do
# Playtime by Level
plays=$(grep "${LEVELS[$i]}.*Level Start" *audit* | wc -l)
playtime=$(grep "${LEVELS[$i]}.*Level" *audit* | sed 's/s//g' | awk '{sum += $5} END {print sum}')
average_playtime=$(awk -v x="$playtime" -v y="$plays" 'BEGIN {print x / y}')
echo "${LEVELS[$i]} Total Plays $plays" | tee -a $PLAYTIME_FILE
echo "${LEVELS[$i]} Total Playtime $playtime" | tee -a $PLAYTIME_FILE
echo "${LEVELS[$i]} Average Playtime $average_playtime" | tee -a $PLAYTIME_FILE
done
total_games=$(grep "Game Start" *audit* | wc -l)
total_game_time=$(grep "Game End\|Game Timeout\|Game Reset" *audit* | sed 's/s//g' | awk '{sum += $4} END {print sum}')
average_game_time=$(awk -v x="$total_game_time" -v y="$total_games" 'BEGIN {print x / y}')
echo "Total Games $total_games" | tee -a $PLAYTIME_FILE
echo "Total Game Time $total_game_time" | tee -a $PLAYTIME_FILE
echo "Average Game Time $average_game_time" | tee -a $PLAYTIME_FILE
mv $SUMMARY_DIR $USERPROFILE/Desktop
echo "Analytic Summary Completed."
sleep 10
The script running in action with data from MAGFest:
Take a minute or two to figure out what’s going on. Essentially we are looping through various arrays that contain the keywords found in our logs files, and use grep along with wc (word count) to tally up our totals. Then we output the totals to corresponding text files. Using this method we can tally up just about anything that we can key off of. So when we said that Logging is the backbone of our analytics, it really is! Without properly formatted log files this summary script would have been much harder to develop.
Utilizing the Analytics in Unity
Out of all the data we collected and summarized, what we find to be the most useful is the coordinate data. Meaning where the Player did the thing. While the totals and times are great for a quick glance as to how players are performing; the coordinate data is what gives us that extra bit of detail as to what exactly needs to be changed. Our Shell Script generates a coordinate file per level, and they look like this:
SB r1 29.4,6.9 SB r1 28.0,5.5 SB r1 19.2,2.0 SB r1 98.8,-18.2 SB r1 28.1,4.3 SB r1 27.5,6.3 SB r1 28.3,6.7 SB r1 28.3,6.7 SB r1 49.3,-4.3 SB r1 69.5,-4.8 SB r1 29.2,6.9 SB r1 33.9,5.3 SB r1 49.0,-4.3 SB r1 49.0,-1.9 SB r1 55.1,-1.3 SB r1 28.8,3.5 SB r1 37.4,5.5 SB r1 30.0,4.6 SB r1 24.8,3.4 SB r1 24.5,3.3 SF r2 50.7,3.3 SF r2 52.6,-2.9 SF r2 132.9,2.3 SF r2 53.0,0.0 SF r2 51.4,-4.3 SF r2 55.0,-3.6 SF r2 48.2,1.8 SF r2 49.6,1.6 SF r2 49.6,-4.0 SF r2 135.0,0.8 SXB r1 95.2,-9.2 SXB r2 55.5,-4.0 SXB r2 54.8,-2.7 SXB r2 85.4,-1.8 SXB r2 48.5,1.7 SXB r2 49.9,-1.4 SXB r2 60.2,-6.1 SXB r2 47.8,1.8 SXB r2 47.8,1.8 SXB r2 47.8,1.8 SXB r2 47.8,1.8 SXF r2 27.0,-4.6 SXF r2 26.5,-1.3 Glyph r1 12.4,1.6 Glyph r1 12.6,2.1 Glyph r1 12.6,2.5 Glyph r1 12.2,1.6 Glyph r1 12.5,2.0 Glyph r1 12.3,1.6 Glyph r1 12.4,1.0 Glyph r1 12.6,2.4 Glyph r1 12.7,2.6 Glyph r1 12.5,2.2 Glyph r1 12.4,2.4 Glyph r1 12.7,2.5 Glyph r1 12.4,2.3 Glyph r1 12.5,2.1 Glyph r1 12.6,2.3 Glyph r1 12.3,2.0 Glyph r1 12.4,2.4 Glyph r1 12.5,2.1 Glyph r1 12.3,2.0 Glyph r1 12.5,2.7 Glyph r1 12.8,2.1 Glyph r1 12.4,2.2 Glyph r1 12.4,2.2 Glyph r1 12.4,1.9 Glyph r1 12.3,1.8 Glyph r1 12.5,2.3 Glyph r1 12.8,2.6 Glyph r1 12.4,2.1 Glyph r1 12.3,2.2 Glyph r2 26.9,-4.0 Glyph r2 50.1,-1.8 Glyph r2 49.4,-4.0 Glyph r2 49.0,-4.0 Glyph r2 49.1,-3.8 Glyph r2 26.5,-4.0 Glyph r2 49.1,-3.8 Glyph r2 49.2,-1.2 Glyph r2 49.6,-1.4 Glyph r2 49.6,-2.3 Glyph r2 48.7,-4.0 Glyph r2 49.9,-2.2 Glyph r2 48.9,-1.9 Glyph r2 49.3,-2.2 Glyph r2 49.6,-1.4 Glyph r2 49.5,-1.7 Glyph r2 49.7,-1.3 Glyph r2 26.1,-4.6 Glyph r2 49.4,-4.0 Glyph r2 49.4,-4.0 Glyph r2 49.6,-1.0 Glyph r2 50.2,-3.4 Glyph r2 48.7,-4.0 Glyph r2 49.8,-4.0 Death r1 12.5,-2.2 Death r1 5.6,-2.4 Death r1 42.5,-24.2 Death r1 97.0,-24.1 Death r1 15.8,-2.4 Death r1 2.6,-1.9 Death r1 73.9,-24.0 Death r1 20.4,-1.4 Death r1 1.9,-2.0 Death r1 12.5,-2.2 Death r1 4.5,-2.5 Death r1 90.2,-23.9 Death r1 7.4,-2.3 Death r1 62.7,-24.2 Death r1 19.8,-1.9 Death r1 20.4,-1.4 Death r1 2.6,-1.9 Death r1 11.0,-2.3 Death r1 14.7,-2.0 Death r1 20.4,-1.4 Death r1 15.8,-2.4 Death r1 100.8,-23.9 Death r1 12.5,-2.2 Death r1 15.2,-2.0 Death r1 12.5,-2.2 Death r1 61.2,-24.1 Death r1 5.8,-2.3 Death r1 127.5,-24.2 Death r1 12.5,-2.2 Death r1 2.5,-1.9 Death r1 20.4,-1.4 Death r1 20.4,-1.4 Death r1 20.4,-1.4 Death r1 23.3,-1.0 Death r1 155.9,-24.2 Death r1 20.4,-1.4 Death r1 21.4,-1.0 Death r1 7.4,-2.3 Death r1 7.4,-2.3 Death r1 13.6,-2.4 Death r1 141.1,-24.1 Death r1 20.4,-1.4 Death r1 20.4,-1.4 Death r1 12.5,-2.2 Death r1 12.5,-2.2 Death r1 2.6,-1.9 Death r1 7.4,-2.3 Death r1 47.7,-24.2 Death r1 12.5,-2.2 Death r1 2.6,-1.9 Death r1 22.6,-1.1 Death r1 45.5,-24.0 Death r1 20.4,-1.4
Then we bring all these coordinate files into Unity under a Resources -> Editor folder
Next we use our custom made “Heatmap” tool, that’s actually more of a Density Scatterplot, that will plot each piece of data for the corresponding level, parenting it to the appropriate row, and giving it the correct icon for the event.
In this Screenshot you can see how it’s definitely a Scatterplot and we call it a Density Scatterplot because we reduced the transparency for each icon. So they get brighter when there’s more stacked. This helps us detect the events of many as opposed to a stray event from a single person or two.
Notice this is plotting all the events for all our different Universes, which can get confusing quick. Luckily, our Script is setup to create the icons in a parenting structure that easily allows us to toggle the events we want to look at on/off.
Now we are just showing Deaths that occurred in the first universe aka r1. In our Hierarchy it looks like this:
It’s great reassurance when we come across an odd or challenging section of a level, and can toggle on the heatmap to see how other players have been performing in the same spot.
To make all this work and have the right icons go the right points, we developed two C# Scripts. The first is used to store the icon, row, and coordinate for each plot point, and the second is an EditorWindow that can load/unload the heatmap on the fly whether we’re in the editor or the game view:
using System;
using UnityEngine;
namespace Heatmap
{
public class HeatmapData
{
public HeatmapIcon Icon { get; private set; }
public int Row { get; private set; }
public Vector2 Coordinate { get; private set; }
public HeatmapData(string data)
{
string[] fields = data.Split(' ');
Icon = (HeatmapIcon) Enum.Parse(typeof(HeatmapIcon), "Heatmap_" + fields[0]);
Row = int.Parse(fields[1].Remove(0, 1));
string[] coordinates = fields[2].Split(',');
Coordinate = new Vector2(float.Parse(coordinates[0]), float.Parse(coordinates[1]));
}
}
}
using EditorExtension;
using System;
using UnityEditor;
using UnityEngine;
using UnityEngine.SceneManagement;
namespace Heatmap
{
public enum HeatmapIcon { Heatmap_Death, Heatmap_SF, Heatmap_SB, Heatmap_SXF, Heatmap_SXB, Heatmap_Glyph };
public class HeatmapManager : EditorWindow
{
private const string LoadDir = "Tools/Load Heatmap %h";
private const string UnloadDir = "Tools/Unload Heatmap %#h";
private const string FileLocation = "Editor/Coordinates/TooManyGames/";
private const string FileLabel = "_coordinates";
private const string RowName = "Row_";
private const string HeatmapRowName = "Heatmap_Row_";
private const int RowCount = 3;
private const int DirectoryGroup = 101;
private static Transform[][] heatmapRows = new Transform[RowCount][];
[MenuItem(LoadDir, false, 101)]
public static void LoadHeatmap()
{
Debug.Log("Loading Heatmap...");
// Load all the icons from Resources folder
int iconCount = Enum.GetNames(typeof(HeatmapIcon)).Length;
GameObject[] icons = new GameObject[iconCount];
for (int i = 0; i < icons.Length; i++)
{
icons[i] = (GameObject) Resources.Load("Heatmap Prefabs/" + ((HeatmapIcon) i).ToString(), typeof(GameObject));
}
string currentSceneName = SceneManager.GetActiveScene().name;
Debug.Log(FileLocation + currentSceneName);
HeatmapData[] heatmapData = LoadHeatmapData(((TextAsset) Resources.Load(FileLocation + currentSceneName + FileLabel, typeof(TextAsset))).text.TrimEnd().Split(new string[] { "\n" }, StringSplitOptions.None));
CreateHeatmapRowObjects(iconCount);
for (int i = 0; i < heatmapData.Length; i++)
{
GameObject icon = Instantiate(icons[(int) heatmapData[i].Icon], heatmapRows[heatmapData[i].Row - 1][(int) heatmapData[i].Icon + 1]);
icon.transform.position = heatmapData[i].Coordinate;
}
}
[MenuItem(UnloadDir, false, DirectoryGroup + 1)]
public static void UnloadHeatmap()
{
Debug.Log("Unloading Heatmap...");
Transform rowManager = GetRowManager();
for (int i = 1; i <= RowCount; i++)
{
Transform row = rowManager.Find(RowName + i);
if (row != null)
{
DestroyImmediate(row.Find(HeatmapRowName + i).gameObject);
}
}
}
private static HeatmapData[] LoadHeatmapData(string[] lines)
{
HeatmapData[] heatmapData = new HeatmapData[lines.Length];
for (int i = 0; i < lines.Length; i++)
{
heatmapData[i] = new HeatmapData(lines[i]);
}
return heatmapData;
}
private static void CreateHeatmapRowObjects(int length)
{
Transform rowManager = GetRowManager();
// +1 for the Heatmap_Row_i parent
length += 1;
for (int row = 0; row < heatmapRows.Length; row++)
{
Transform rowTrans = rowManager.Find(RowName + (row + 1));
if (rowTrans != null)
{
CreateHeatmapIconObjects(length, row, rowTrans);
}
}
}
private static void CreateHeatmapIconObjects(int length, int row, Transform rowTrans)
{
heatmapRows[row] = new Transform[length];
for (int k = 0; k < heatmapRows[row].Length; k++)
{
if (k == 0)
{
heatmapRows[row][k] = new GameObject(HeatmapRowName + (row + 1)).transform;
heatmapRows[row][k].parent = rowTrans;
}
else
{
heatmapRows[row][k] = new GameObject(((HeatmapIcon) k - 1).ToString()).transform;
heatmapRows[row][k].parent = heatmapRows[row][0];
}
}
}
private static Transform GetRowManager()
{
return GameObject.Find("RowManager_" + SceneManager.GetActiveScene().name).transform;
}
}
}
This second script handles the bulk of the logic. Parsing the text files, creating the plot points, loading the correct icons, and then scattering the plot points across the level in an organized hierarchical fashion.
A lot of design, testing, and effort goes into making sure each level in ParaLily is an exciting, rewarding, and fair experience. With this method of logging everything/anything, summarizing it up, and bringing it all back in Unity; we can ensure our levels meet our high quality standards!
Thank you,
ParaLily Dev Team
Nate & Jeff
P.S. We skimmed over a lot of the technical setup and coding portions, so if you have any questions or comments feel free to email us info@ParaLilyGame.com and we'll get back to you when we can!