ParaLily Update #4: Improving Level Design with Player Analytics and Heatmaps
In our last update we showed a bunch of Player Analytics that we collected from MAGFest including playtime, death variations, levels beaten, etc. Now we’d like to share how we collected, summarized, and utilized all that data to help improve ParaLily.
As a two man team we spend a lot of time not only designing and creating ParaLily, but also playing testing, ParaLily. Unfortunately all that testing really skews our perception of difficulty within the game. In order to combat this we have to let people test our game and see for ourselves where things are too hard or confusing. Fortunately conventions are great place to get feedback! We can watch players firsthand, and see where they get stuck or lost. Even better, we can make sure we don’t miss a beat, by keeping track of our players events in an audit log file.
Logging
The backbone of our analytic system relies heavily on Logging. Specifically Audit Logging, which is a method of recording chronological events. For an everyday example someone might log when they ate an apple. It’s important to note that we don’t care how many apples they ate, we just log each one as a separate entry like so:
- ate apple
- ate apple
- ate apple
Now if we want to know where each apple was eaten, or what kind of specific apple it was, then we just add that to our log entry:
- ate apple fuji kitchen
- ate apple gala office
- ate apple fuji bedroom
Looking at the logs we can see that three apples were eaten. Two of which were fuji, and were eaten at home. So from this bit of data we can assume that at home there are fuji apples and at work there are gala apples.
In ParaLily we apply this same concept where each line in our log file is simply an event that the Player executed. It ends up looking something like this:
- 04d17h27m00s Game Start 0f4fbaf6-9e9a-4740-a01a-86369f74419e
- 04d17h27m24s 1-1 Level Start
- 04d17h27m48s 1-1 r1 81.6,7.7 Glyph
- 04d17h27m54s 1-1 r1 131.4,1.3 Glyph
- 04d17h28m00s 1-1 Level Win 35.92s
- 04d17h28m01s 1-2 Level Start
- 04d17h28m08s 1-2 r1 -4.8,-5.0 SXB
- 04d17h28m08s 1-2 r1 -0.3,-24.3 Death Fall
- 04d17h28m09s 1-2 Level Lose 7.45s
- 04d17h28m10s 1-2 Level Start
- 04d17h28m14s 1-2 r1 -25.8,-1.9 SB
- 04d17h28m31s 1-2 r2 16.6,-3.6 SF
- 04d17h28m35s 1-2 r1 16.9,-3.2 Glyph
- 04d17h28m37s 1-2 r1 16.9,-0.1 SB
- 04d17h28m42s 1-2 r2 33.9,0.5 SF
- 04d17h28m47s 1-2 r1 63.8,-2.1 SB
- 04d17h28m53s 1-2 r2 87.9,0.3 SF
- 04d17h28m54s 1-2 r1 89.9,-3.0 Death Thorn
- 04d17h28m54s 1-2 Level Lose 44.25s
- 04d17h28m56s 1-2 Level Start
- 04d17h29m02s 1-2 r1 -19.2,-0.3 SB
- 04d17h29m11s 1-2 r2 35.1,-1.1 SF
- 04d17h29m17s 1-2 r1 65.8,0.0 SB
- 04d17h29m22s 1-2 r2 71.6,-5.8 SF
- 04d17h29m23s 1-2 r1 71.6,-5.3 Glyph
- 04d17h29m25s 1-2 r1 75.9,-3.4 SB
- 04d17h29m27s 1-2 r2 82.4,-2.6 SF
- 04d17h29m31s 1-2 r1 93.6,-4.5 SB
- 04d17h29m32s 1-2 r2 98.2,-20.5 SF
- 04d17h29m33s 1-2 r1 98.9,-24.1 Death Fall
- 04d17h29m33s 1-2 Level Lose 37.52s
- 04d17h29m35s 1-2 Level Start
- 04d17h29m39s 1-2 r1 -23.6,-1.1 SB
- 04d17h29m49s 1-2 r2 41.3,-0.6 SF
- 04d17h30m00s 1-2 r1 101.8,-1.3 SB
- 04d17h30m06s 1-2 r2 130.2,1.1 SF
- 04d17h30m10s 1-2 Level Win 35.15s
- 04d17h30m11s 1-3 Level Start
- 04d17h30m19s 1-3 r1 19.8,1.8 Glyph
- 04d17h30m22s 1-3 r1 31.3,0.7 SB
- 04d17h30m31s 1-3 r2 61.2,-0.3 SB
- 04d17h30m42s 1-3 r3 90.8,-6.7 Glyph
- 04d17h30m43s 1-3 r3 94.6,-9.6 SF
- 04d17h30m47s 1-3 r2 110.6,-2.5 Glyph
- 04d17h30m52s 1-3 r2 146.5,-1.3 SB
- 04d17h30m54s 1-3 r3 151.2,-10.9 SF
- 04d17h30m55s 1-3 r2 152.6,-24.2 Death Fall
- 04d17h30m55s 1-3 Level Lose 43.77s
- 04d17h30m57s 1-3 Level Start
- 04d17h31m06s 1-3 r1 31.4,0.4 SB
- 04d17h31m13s 1-3 r2 59.8,0.0 SB
- 04d17h31m21s 1-3 r3 98.0,-12.4 SXF
- 04d17h31m21s 1-3 r3 98.1,-12.9 Death LavaFlow
- 04d17h31m21s 1-3 Level Lose 24.46s
- 04d17h31m23s 1-3 Level Start
- 04d17h31m32s 1-3 r1 31.3,-0.1 SB
- 04d17h31m41s 1-3 r2 59.4,-0.5 SB
- 04d17h31m49s 1-3 r3 95.2,-7.3 SF
- 04d17h32m02s 1-3 r2 154.0,-24.2 Death Fall
- 04d17h32m03s 1-3 Level Lose 40.07s
- 04d17h32m04s 1-3 Level Start
- 04d17h32m14s 1-3 r1 31.8,0.0 SB
- 04d17h32m19s 1-3 r2 59.1,-2.0 SF
- 04d17h32m20s 1-3 r1 59.2,-2.3 SB
- 04d17h32m22s 1-3 r2 64.4,-0.2 SF
- 04d17h32m24s 1-3 r1 67.0,-24.3 Death Fall
- 04d17h32m24s 1-3 Level Lose 20.14s
- 04d17h32m26s 1-3 Level Start
- 04d17h32m35s 1-3 r1 30.4,0.4 SB
- 04d17h32m42s 1-3 r2 60.6,-0.5 SB
- 04d17h32m49s 1-3 r3 91.5,-7.1 SF
- 04d17h33m00s 1-3 r2 145.0,-1.8 SF
- 04d17h33m01s 1-3 r1 146.7,-1.0 SB
- 04d17h33m02s 1-3 r2 148.8,-1.7 SF
- 04d17h33m05s 1-3 Level Win 39.18s
- 04d17h33m06s 1-4 Level Start
- 04d17h33m13s 1-4 r1 14.4,2.4 Glyph
- 04d17h33m17s 1-4 r1 34.2,5.0 SB
- 04d17h33m22s 1-4 r2 49.2,1.6 SB
- 04d17h33m24s 1-4 r3 49.2,2.0 SF
- 04d17h33m25s 1-4 r2 49.2,1.6 SF
- 04d17h33m26s 1-4 r1 49.2,0.4 SXB
- 04d17h33m31s 1-4 r1 71.5,-24.2 Death Fall
- 04d17h33m32s 1-4 Level Lose 25.30s
- 04d17h33m33s 1-4 Level Start
- 04d17h33m43s 1-4 r1 34.8,4.9 SB
- 04d17h33m46s 1-4 r2 48.3,1.8 SF
- 04d17h33m49s 1-4 r1 49.2,-1.2 SXB
- 04d17h33m50s 1-4 r1 49.2,-1.6 SB
- 04d17h33m51s 1-4 r2 49.2,-1.6 Glyph
- 04d17h33m54s 1-4 r2 51.7,-4.4 SF
- 04d17h33m56s 1-4 r1 55.9,-1.3 SB
- 04d17h34m00s 1-4 r2 51.5,3.7 SB
- 04d17h34m24s 1-4 r3 80.2,-5.8 SF
- 04d17h34m25s 1-4 r2 80.2,-7.0 SF
- 04d17h34m26s 1-4 r1 80.2,-12.5 SXB
- 04d17h34m26s 1-4 r1 80.2,-24.2 Death Fall
- 04d17h34m27s 1-4 Level Lose 53.39s
- 04d17h34m28s 1-4 Level Start
- 04d17h34m36s 1-4 r1 29.0,6.0 SB
- 04d17h34m40s 1-4 r2 53.1,3.5 SB
- 04d17h34m46s 1-4 r3 79.6,-9.2 SF
- 04d17h34m49s 1-4 r2 87.7,3.2 SB
- 04d17h34m51s 1-4 r3 84.3,3.3 Glyph
- 04d17h34m52s 1-4 r3 88.0,2.6 SF
- 04d17h34m59s 1-4 r2 134.0,2.6 SF
- 04d17h35m02s 1-4 Level Win 33.86s
- 04d17h35m03s 1-5 Level Start
- 04d17h35m11s 1-5 r1 -78.5,-1.2 Death Bunny
- 04d17h35m11s 1-5 Level Lose 7.72s
- 04d17h35m13s 1-5 Level Start
- 04d17h35m30s 1-5 r1 -60.4,2.3 SB
- 04d17h35m35s 1-5 r2 -43.6,2.1 SB
- 04d17h35m36s 1-5 r3 -38.5,-12.0 Death LavaFlow
- 04d17h35m37s 1-5 Level Lose 24.10s
- 04d17h35m38s 1-5 Level Start
- 04d17h35m48s 1-5 r1 -59.9,2.1 SB
- 04d17h35m52s 1-5 r2 -43.3,2.1 SF
- 04d17h35m53s 1-5 r1 -41.8,0.5 SB
- 04d17h35m55s 1-5 r2 -45.5,1.6 SF
- 04d17h35m56s 1-5 r1 -45.5,1.6 Glyph
- 04d17h35m56s 1-5 r1 -45.4,1.1 SB
- 04d17h35m57s 1-5 r2 -44.0,-5.8 SB
- 04d17h35m58s 1-5 r3 -42.8,-12.1 Death LavaFlow
- 04d17h35m58s 1-5 Level Lose 20.09s
- 04d17h36m00s 1-5 Level Start
- 04d17h36m05s 1-5 r1 -86.8,-3.0 Death Bunny
- 04d17h36m05s 1-5 Level Lose 5.64s
- 04d17h36m07s 1-5 Level Start
- 04d17h36m16s 1-5 r1 -60.7,2.1 SB
- 04d17h36m19s 1-5 r2 -43.9,2.1 SB
- 04d17h36m20s 1-5 r3 -42.0,1.4 SF
- 04d17h36m22s 1-5 r2 -34.1,3.5 SB
- 04d17h36m23s 1-5 r3 -31.2,3.6 SF
- 04d17h36m24s 1-5 r2 -29.4,0.6 SF
- 04d17h36m27s 1-5 r1 -14.2,-1.6 Glyph
- 04d17h36m28s 1-5 r1 -12.7,-1.6 SB
- 04d17h36m32s 1-5 r2 1.5,2.3 SF
- 04d17h36m33s 1-5 r1 6.0,-0.7 SB
- 04d17h36m34s 1-5 r2 7.1,-3.6 SB
- 04d17h36m35s 1-5 r3 7.5,-5.5 Death SkullSpider
- 04d17h36m35s 1-5 Level Lose 28.16s
- 04d17h36m37s 1-5 Level Start
- 04d17h36m46s 1-5 r1 -60.0,2.1 SB
- 04d17h36m49s 1-5 r2 -42.8,1.7 SB
- 04d17h36m50s 1-5 r3 -40.7,-1.5 SXF
- 04d17h36m50s 1-5 r3 -37.7,-12.1 Death LavaFlow
- 04d17h36m50s 1-5 Level Lose 13.77s
- 04d17h36m52s 1-5 Level Start
- 04d17h37m01s 1-5 r1 -59.1,1.7 SB
- 04d17h37m05s 1-5 r2 -34.9,3.4 SF
- 04d17h37m08s 1-5 r1 -14.7,-1.1 SB
- 04d17h37m12s 1-5 r2 1.1,2.3 SB
- 04d17h37m14s 1-5 r3 6.6,-5.6 Death SkullSpider
- 04d17h37m14s 1-5 Level Lose 22.10s
- 04d17h37m15s 1-5 Level Start
- 04d17h37m24s 1-5 r1 -59.8,1.7 SB
- 04d17h37m28s 1-5 r2 -36.3,2.7 SXF
- 04d17h37m29s 1-5 r2 -31.8,-1.3 SF
- 04d17h37m32s 1-5 r1 -14.0,-1.4 SB
- 04d17h37m36s 1-5 r2 8.1,-0.6 SB
- 04d17h37m37s 1-5 r3 9.7,-6.0 Death SkullSpider
- 04d17h37m37s 1-5 Level Lose 22.04s
- 04d17h37m39s 1-5 Level Start
- 04d17h37m48s 1-5 r1 -58.7,1.4 SB
- 04d17h37m52s 1-5 r2 -36.3,2.6 SXF
- 04d17h37m52s 1-5 r2 -32.8,-0.6 SF
- 04d17h37m56s 1-5 r1 -14.0,-1.1 SB
- 04d17h37m59s 1-5 r2 3.0,4.0 SF
- 04d17h38m00s 1-5 r1 4.1,4.2 SB
- 04d17h38m01s 1-5 r2 7.1,1.5 SB
- 04d17h38m02s 1-5 r3 9.7,-5.8 Death SkullSpider
- 04d17h38m02s 1-5 Level Lose 23.44s
- 04d17h38m04s 1-5 Level Start
- 04d17h38m13s 1-5 r1 -59.6,1.9 SB
- 04d17h38m17s 1-5 r2 -34.6,3.4 SF
- 04d17h38m20s 1-5 r1 -14.1,-1.0 SB
- 04d17h38m23s 1-5 r2 0.3,3.5 SB
- 04d17h38m24s 1-5 r3 1.8,4.0 SF
- 04d17h38m25s 1-5 r2 3.1,3.6 SB
- 04d17h38m26s 1-5 r3 7.3,-5.8 Death SkullSpider
- 04d17h38m27s 1-5 Level Lose 22.79s
- 04d17h38m28s 1-5 Level Start
- 04d17h38m37s 1-5 r1 -61.9,1.5 SB
- 04d17h38m41s 1-5 r2 -35.9,3.2 SXF
- 04d17h38m42s 1-5 r2 -30.9,-1.8 SF
- 04d17h38m45s 1-5 r1 -14.0,-1.1 SB
- 04d17h38m49s 1-5 r2 2.2,4.1 SB
- 04d17h38m53s 1-5 r3 17.6,-0.9 SF
- 04d17h38m55s 1-5 r2 22.8,0.0 SF
- 04d17h38m56s 1-5 r1 20.9,-24.3 Death Fall
- 04d17h38m57s 1-5 Level Lose 28.61s
- 04d17h38m58s 1-5 Level Start
- 04d17h39m07s 1-5 r1 -58.6,1.4 SB
- 04d17h39m12s 1-5 r2 -33.3,2.0 SF
- 04d17h39m15s 1-5 r1 -15.5,-1.4 SB
- 04d17h39m18s 1-5 r2 0.5,-0.3 SF
- 04d17h39m19s 1-5 r1 2.8,2.3 SB
- 04d17h39m21s 1-5 r2 6.0,-3.4 SB
- 04d17h39m22s 1-5 r3 4.8,-6.2 Death SkullSpider
- 04d17h39m22s 1-5 Level Lose 24.21s
- 04d17h39m24s 1-5 Level Start
- 04d17h39m33s 1-5 r1 -59.2,1.8 SB
- 04d17h39m37s 1-5 r2 -33.7,1.6 SF
- 04d17h39m41s 1-5 r1 -13.5,-1.1 SB
- 04d17h39m44s 1-5 r2 0.0,2.9 SF
- 04d17h39m45s 1-5 r1 1.0,3.8 SB
- 04d17h39m46s 1-5 r2 3.8,-5.2 SB
- 04d17h39m50s 1-5 r3 16.0,-1.6 SF
- 04d17h39m52s 1-5 r2 20.9,3.1 SB
- 04d17h39m55s 1-5 r3 30.1,2.9 SF
- 04d17h39m57s 1-5 r2 34.5,3.7 SB
- 04d17h39m59s 1-5 r3 40.3,3.7 SF
- 04d17h40m04s 1-5 r2 38.1,8.3 SB
- 04d17h40m06s 1-5 r3 38.2,-5.4 SF
- 04d17h40m07s 1-5 r2 38.8,-6.2 SB
- 04d17h40m09s 1-5 r3 38.5,-7.5 SF
- 04d17h40m10s 1-5 r2 37.8,-6.3 Glyph
- 04d17h40m11s 1-5 r2 37.7,-3.3 SF
- 04d17h40m12s 1-5 r1 37.8,-0.8 SB
- 04d17h40m16s 1-5 r2 56.0,3.2 SB
- 04d17h40m17s 1-5 r3 59.1,4.6 SF
- 04d17h40m19s 1-5 r2 62.1,1.1 SF
- 04d17h40m25s 1-5 r1 99.8,-24.3 Death Fall
- 04d17h40m25s 1-5 Level Lose 60.89s
- 04d17h40m26s 1-5 Level Start
- 04d17h40m35s 1-5 r1 -59.3,1.8 SB
- 04d17h40m40s 1-5 r2 -35.4,3.1 SF
- 04d17h40m43s 1-5 r1 -15.4,-1.2 SB
- 04d17h40m46s 1-5 r2 0.2,1.6 SB
- 04d17h40m49s 1-5 r3 4.0,-5.8 Death SkullSpider
- 04d17h40m49s 1-5 Level Lose 22.35s
- 04d17h40m50s 1-5 Level Start
- 04d17h40m59s 1-5 r1 -59.8,2.1 SB
- 04d17h41m03s 1-5 r2 -35.5,3.1 SF
- 04d17h41m07s 1-5 r1 -14.5,-1.0 SB
- 04d17h41m10s 1-5 r2 -2.1,2.9 SF
- 04d17h41m11s 1-5 r1 3.0,-2.0 SB
- 04d17h41m12s 1-5 r2 3.7,-14.2 SB
- 04d17h41m13s 1-5 r3 3.7,-14.2 Death LavaFlow
- 04d17h41m13s 1-5 Level Lose 23.19s
- 04d17h41m15s 1-5 Level Start
- 04d17h41m24s 1-5 r1 -59.4,1.8 SB
- 04d17h41m28s 1-5 r2 -33.3,-0.6 SF
- 04d17h41m32s 1-5 r1 -13.6,-1.2 SB
- 04d17h41m35s 1-5 r2 0.7,2.3 SB
- 04d17h41m40s 1-5 r3 18.0,-3.3 SF
- 04d17h41m42s 1-5 r2 20.5,3.4 SB
- 04d17h41m44s 1-5 r3 27.0,3.6 SF
- 04d17h41m50s 1-5 r2 56.4,1.8 SF
- 04d17h42m00s 1-5 Level Win 45.15s
- 04d17h42m02s 1-6 Level Start
- 04d17h42m14s 1-6 r1 34.6,-3.5 Death Bunny
- 04d17h42m14s 1-6 Level Lose 11.95s
- 04d17h42m16s 1-6 Level Start
- 04d17h42m29s 1-6 r1 49.8,-2.0 SB
- 04d17h42m31s 1-6 r2 46.9,1.6 Death Creep
- 04d17h42m31s 1-6 Level Lose 15.42s
- 04d17h42m33s 1-6 Level Start
- 04d17h42m44s 1-6 r1 35.5,2.8 SB
- 04d17h42m51s 1-6 r2 76.2,0.8 SB
- 04d17h42m55s 1-6 r3 98.4,1.3 SF
- 04d17h42m59s 1-6 r2 120.3,-8.4 SB
- 04d17h43m00s 1-6 r3 120.7,-10.0 SXF
- 04d17h43m00s 1-6 r3 120.9,-12.1 Death LavaFlow
- 04d17h43m01s 1-6 Level Lose 27.53s
- 04d17h43m02s 1-6 Level Start
- 04d17h43m14s 1-6 r1 35.5,2.8 SB
- 04d17h43m21s 1-6 r2 76.8,1.0 SB
- 04d17h43m25s 1-6 r3 96.4,0.9 SF
- 04d17h43m31s 1-6 r2 137.1,0.5 SF
- 04d17h43m33s 1-6 r1 152.8,0.5 SB
- 04d17h43m35s 1-6 r2 160.0,0.4 SB
- 04d17h43m38s 1-6 r3 175.8,0.2 SF
- 04d17h43m42s 1-6 r2 197.3,1.1 SB
- 04d17h43m44s 1-6 r3 211.2,-2.7 SF
- 04d17h43m46s 1-6 r2 220.6,1.2 SB
- 04d17h43m52s 1-6 r3 250.8,2.7 Death LavaPool
- 04d17h43m52s 1-6 Level Lose 49.95s
- 04d17h43m54s 1-6 Level Start
- 04d17h44m06s 1-6 r1 36.4,2.8 SB
- 04d17h44m12s 1-6 r2 76.2,1.1 SB
- 04d17h44m16s 1-6 r3 96.5,0.9 SF
- 04d17h44m22s 1-6 r2 135.2,2.5 SF
- 04d17h44m25s 1-6 r1 152.9,0.7 SB
- 04d17h44m27s 1-6 r2 159.1,1.0 SB
- 04d17h44m30s 1-6 r3 175.5,1.4 SF
- 04d17h44m33s 1-6 r2 197.8,1.1 SB
- 04d17h44m36s 1-6 r3 211.5,-2.5 SF
- 04d17h44m38s 1-6 r2 221.8,2.8 SB
- 04d17h44m41s 1-6 r3 237.5,-3.0 Death SkullSpider
- 04d17h44m42s 1-6 Level Lose 47.46s
- 04d17h44m43s 1-6 Level Start
- 04d17h44m55s 1-6 r1 36.8,2.8 SB
- 04d17h45m02s 1-6 r2 76.6,1.0 SB
- 04d17h45m05s 1-6 r3 96.6,1.5 SF
- 04d17h45m12s 1-6 r2 136.3,2.6 SF
- 04d17h45m15s 1-6 r1 153.6,0.7 SB
- 04d17h45m17s 1-6 r2 161.1,1.3 SB
- 04d17h45m20s 1-6 r3 175.3,1.6 SF
- 04d17h45m23s 1-6 r2 197.1,1.0 SB
- 04d17h45m26s 1-6 r3 212.5,-3.0 SF
- 04d17h45m27s 1-6 r2 218.5,0.9 SB
- 04d17h45m32s 1-6 r3 231.1,-2.1 Death Creep
- 04d17h45m32s 1-6 Level Lose 48.82s
- 04d17h45m34s 1-6 Level Start
- 04d17h45m46s 1-6 r1 37.1,2.7 SB
- 04d17h45m52s 1-6 r2 76.3,1.0 SB
- 04d17h45m55s 1-6 r3 98.1,1.7 SF
- 04d17h46m01s 1-6 r2 135.6,2.6 SF
- 04d17h46m04s 1-6 r1 151.5,0.1 SB
- 04d17h46m06s 1-6 r2 160.4,1.3 SB
- 04d17h46m08s 1-6 r3 172.9,2.8 SF
- 04d17h46m12s 1-6 r2 197.5,0.9 SB
- 04d17h46m15s 1-6 r3 210.6,-2.4 SF
- 04d17h46m17s 1-6 r2 221.9,2.0 SB
- 04d17h46m25s 1-6 r3 254.8,5.2 Glyph
- 04d17h46m31s 1-6 r3 291.6,-5.6 SF
- 04d17h46m35s 1-6 r2 316.3,-2.3 SF
- 04d17h46m38s 1-6 Level Win 63.54s
- 04d17h46m38s Game End 1177.96s 0f4fbaf6-9e9a-4740-a01a-86369f74419e
So what’s going on in all this mess? Just like in the apple scenario a space is our delimiter for each column. So our columns are roughly:
- DATE/TIME LEVEL ROW PLAYER_POSITION EVENT
Date and time are great to have because it let’s us see just how long it took players to go from one place to another. Then in ParaLily we not only have levels like world 1 level 1, but also Universes or Rows as we call them, which are r1, r2, and r3. Logging the Players exact position gives us a lot of opportunities like generating heatmaps. Lastly, we log the event itself: Death, Shift Forward, Shift Back, or Glyph Collected.
We also have special scenarios like the game starting or ending, a level starting or ending. Timestamps for key things like when the game ends, in this case they beat it after 1177.96s. A Unique Identifier is generated per play-through, and helps us find specific logs if something crazy happens in game.
Then when the next player comes along we create a new log file. So each log file itself is a single play-through. Our folder by the end of a convention:
For developers looking to setup their own loggers it’s pretty straightforward. My biggest tip would be to make sure you are not re-opening the log file every time you write to it. Here’s the code we used for our Audit Logger:
- using System;
- using System.IO;
- using System.Text;
- using UnityEngine;
- public static class Logger
- {
- private static readonly string[] Row = new string[] { " r1 ", " r2 ", " r3 " };
- private static readonly string LogDir = Path.Combine(Application.persistentDataPath, "Logs/");
- private static readonly string FileName = "_ParaLily_audit.log";
- private static readonly string FileDateTimeFormat = "yyyy-MM-dd'T'HH'h'mm'm'ss's'";
- private static readonly string LogDateTimeFormat = "dd'd'HH'h'mm'm'ss's'";
- public static Guid Uid { get; set; }
- private static string auditLogPath;
- private static float gameStart;
- private static float playStart;
- private static StreamWriter logWriter;
- static Logger()
- {
- SetNewGame();
- if (!Directory.Exists(LogDir))
- {
- Directory.CreateDirectory(LogDir);
- }
- }
- public static void AuditGame()
- {
- SetNewGame();
- StringBuilder sb = new StringBuilder();
- sb.Append(DateTime.Now.ToString(LogDateTimeFormat)).Append(" Game Start ").Append(Uid.ToString());
- WriteLog(sb.ToString());
- }
- public static void AuditGame(string text)
- {
- StringBuilder sb = new StringBuilder();
- sb.Append(DateTime.Now.ToString(LogDateTimeFormat)).Append(" Game ").Append(text).Append(" ")
- .Append(GameDurationInSecondsToString()).Append(" ").Append(Uid.ToString());
- WriteLog(sb.ToString());
- }
- public static void AuditLevelStart()
- {
- StringBuilder sb = new StringBuilder();
- playStart = Time.time;
- sb.Append(DateTime.Now.ToString(LogDateTimeFormat)).Append(" ").Append(GameManager.instance.CurrentSceneName).Append(" Level Start");
- WriteLog(sb.ToString());
- }
- public static void AuditLevel(string text)
- {
- StringBuilder sb = new StringBuilder();
- sb.Append(DateTime.Now.ToString(LogDateTimeFormat)).Append(" ").Append(GameManager.instance.CurrentSceneName).Append(" Level ").Append(text).Append(" ").Append(PlayDurationInSecondsToString());
- WriteLog(sb.ToString());
- }
- public static void AuditFull(string text)
- {
- StringBuilder sb = new StringBuilder();
- sb.Append(DateTime.Now.ToString(LogDateTimeFormat)).Append(" ").Append(GameManager.instance.CurrentSceneName).Append(Row[RowManager.instance.CurrentParallel])
- .Append(PlayerController.instance.transform.position.ToStringXY()).Append(" ").Append(text);
- WriteLog(sb.ToString());
- }
- private static void WriteLog(string text)
- {
- logWriter.WriteLine(text);
- logWriter.Flush();
- }
- public static float GameDurationInSeconds()
- {
- return Time.time - gameStart;
- }
- public static float PlayDurationInSeconds()
- {
- return Time.time - playStart;
- }
- private static string GameDurationInSecondsToString()
- {
- return GameDurationInSeconds().ToString("0.00s");
- }
- private static string PlayDurationInSecondsToString()
- {
- return PlayDurationInSeconds().ToString("0.00s");
- }
- private static void SetNewGame()
- {
- Uid = Guid.NewGuid();
- gameStart = Time.time;
- auditLogPath = Path.Combine(LogDir, DateTime.Now.ToString(FileDateTimeFormat) + FileName);
- Dispose();
- logWriter = new StreamWriter(auditLogPath);
- logWriter.AutoFlush = false;
- }
- public static void Dispose()
- {
- logWriter?.Dispose();
- }
- }
Summarizing with Shell Scripting
So what do we do with all of this data? There’s hundreds of lines per file and hundreds of files. It would take forever to sift through and tally all this up by hand. Fear not! Scripting is here to save us from all that misery. To get all the important bits of data we want, we created a shell script to do all the heavy lifting, and generate a clean summarized output of all that player data. After it runs it leaves us with a folder with all of these files:
The files each look similar to this:
- Game Start 109
- Game End 7
- Game Timeout 59
- Game Reset 27
- Level Start 1707
- Level Win 191
- Level Lose 1427
- SB 2893
- SF 1996
- SXB 322
- SXF 360
- Glyph 576
- Death 1427
- Death Thorn 373
- Death Fall 790
- Death SkullSpider 57
- Death LavaPool 24
- Death LavaFlow 116
- Death Creep 12
- Death Bunny 16
- Death Quicksand 39
*Data collected from Cecil Con
And here’s the Shell Script we developed to do all the summarizing:
- #!/bin/bash
- function date_time_format {
- date "+%Y-%m-%d"
- }
- SUMMARY_DIR="summary_$(date_time_format)"
- LEVELS=( "1-1" "1-2" "1-3" "1-4" "1-5" "1-6" "1-7" "1-8" )
- ROWS=( "r1" "r2" "r3" )
- DAYS=( "14d" ) # Days the event ran
- HOURS=( "10h" "11h" "12h" "13h" "14h" "15h" "16h" "17h" )
- #HOURS=( "00h" "01h" "02h" "03h" "04h" "05h" "06h" "07h" "08h" "09h" "10h" "11h" "12h" "13h" "14h" "15h" "16h" "17h" "18h" "19h" "20h" "21h" "22h" "23h" )
- KEYWORDS_BY_LEVEL_BY_ROW=( "SB" "SF" "SXB" "SXF" "Glyph" "Death" )
- KEYWORDS_BY_LEVEL=( "Level Start" "Level Win" "Level Lose" "Level Reset" "Level Timeout" "SB" "SF" "SXB" "SXF" "Glyph" "Death" "Death Thorn" "Death Fall" "Death SkullSpider" "Death LavaPool" "Death LavaFlow" "Death Creep" "Death Bunny" "Death Quicksand" )
- KEYWORDS_BY_ROW=( "SB" "SF" "SXB" "SXF" "Glyph" "Death" )
- KEYWORDS_OVERALL=( "Game Start" "Game End" "Game Timeout" "Game Reset" "Level Start" "Level Win" "Level Lose" "SB" "SF" "SXB" "SXF" "Glyph" "Death" "Death Thorn" "Death Fall" "Death SkullSpider" "Death LavaPool" "Death LavaFlow" "Death Creep" "Death Bunny" "Death Quicksand" )
- KEYWORDS_BY_COORDINATE=( "SB" "SF" "SXB" "SXF" "Glyph" "Death" )
- KEYWORDS_BY_HOUR=( "Game Start" "Game End" "Game Timeout" "Game Reset" "Level Start" "Level Win" "Level Lose" "SB" "SF" "SXB" "SXF" "Glyph" "Death" "Death Thorn" "Death Fall" "Death SkullSpider" "Death LavaPool" "Death LavaFlow" "Death Creep" "Death Bunny" "Death Quicksand" )
- cd '.\Logs'
- mkdir $SUMMARY_DIR
- echo "Building Analytic Summary to $SUMMARY_DIR..."
- #--Totals by Level by Row Summary--
- TOTALS_BY_LEVEL_BY_ROW_FILE="$SUMMARY_DIR\\totals_by_level_by_row.txt"
- echo "Generating File $TOTALS_BY_LEVEL_BY_ROW_FILE..."
- for (( i=0; i<${#LEVELS[@]}; i++ ))
- do
- for (( k=0; k<${#ROWS[@]}; k++ ))
- do
- for (( j=0; j<${#KEYWORDS_BY_LEVEL_BY_ROW[@]}; j++ ))
- do
- COUNT[$j]=$(grep "${LEVELS[$i]}.*${ROWS[$k]}.*${KEYWORDS_BY_LEVEL_BY_ROW[$j]}" *audit* | wc -l )
- echo "${LEVELS[$i]} ${ROWS[$k]} ${KEYWORDS_BY_LEVEL_BY_ROW[$j]} ${COUNT[$j]}" | tee -a $TOTALS_BY_LEVEL_BY_ROW_FILE
- done
- done
- done
- #--Totals by Row Summary--
- TOTALS_BY_ROW_FILE="$SUMMARY_DIR\\totals_by_row.txt"
- echo "Generating File $TOTALS_BY_ROW_FILE..."
- for (( i=0; i<${#ROWS[@]}; i++ ))
- do
- for (( k=0; k<${#KEYWORDS_BY_ROW[@]}; k++ ))
- do
- COUNT[$k]=$(grep "${ROWS[$i]}.*${KEYWORDS_BY_ROW[$k]}" *audit* | wc -l )
- echo "${ROWS[$i]} ${KEYWORDS_BY_ROW[$k]} ${COUNT[$k]}" | tee -a $TOTALS_BY_ROW_FILE
- done
- done
- #--Totals by Level Summary--
- TOTALS_BY_LEVEL_FILE="$SUMMARY_DIR\\totals_by_level.txt"
- echo "Generating File $TOTALS_BY_LEVEL_FILE..."
- for (( i=0; i<${#LEVELS[@]}; i++ ))
- do
- for (( k=0; k<${#KEYWORDS_BY_LEVEL[@]}; k++ ))
- do
- COUNT[$k]=$(grep "${LEVELS[$i]}.*${KEYWORDS_BY_LEVEL[$k]}" *audit* | wc -l )
- echo "${LEVELS[$i]} ${KEYWORDS_BY_LEVEL[$k]} ${COUNT[$k]}" | tee -a $TOTALS_BY_LEVEL_FILE
- done
- done
- #--Totals Overall Summary--
- TOTALS_FILE="$SUMMARY_DIR\\totals_overall.txt"
- echo "Generating File $TOTALS_FILE..."
- for (( i=0; i<${#KEYWORDS_OVERALL[@]}; i++ ))
- do
- COUNT[$i]=$(grep "${KEYWORDS_OVERALL[$i]}" *audit* | wc -l )
- echo "${KEYWORDS_OVERALL[$i]} ${COUNT[$i]}" | tee -a $TOTALS_FILE
- done
- #--Coordinate Summary--
- echo "Generating Coordinate Files..."
- for (( i=0; i<${#LEVELS[@]}; i++ ))
- do
- coordinate_file="$SUMMARY_DIR\\${LEVELS[$i]}_coordinates.txt"
- for (( k=0; k<${#KEYWORDS_BY_COORDINATE[@]}; k++ ))
- do
- for (( j=0; j<${#ROWS[@]}; j++ ))
- do
- grep "${LEVELS[$i]} ${ROWS[j]}.*${KEYWORDS_BY_COORDINATE[$k]}" *audit* | awk '{print $5,$3,$4;}' >> "$coordinate_file"
- done
- done
- echo "Generated $coordinate_file."
- done
- #--Hourly Totals Summary--
- HOURLY_FILE="$SUMMARY_DIR\\totals_by_day_by_hour.txt"
- echo "Generating File $HOURLY_FILE..."
- for (( i=0; i<${#DAYS[@]}; i++))
- do
- for (( k=0; k<${#HOURS[@]}; k++ ))
- do
- for (( j=0; j<${#KEYWORDS_BY_HOUR[@]}; j++ ))
- do
- count=$(grep "${DAYS[$i]}${HOURS[$k]}.*${KEYWORDS_BY_HOUR[$j]}" *audit* | wc -l)
- echo "${DAYS[$i]}${HOURS[$k]} ${KEYWORDS_BY_HOUR[$j]} $count" | tee -a "$HOURLY_FILE"
- done
- # Game time by Day by Hour
- hourly_game_time=$(grep "${DAYS[$i]}${HOURS[$k]}.*Game End\|${DAYS[$i]}${HOURS[$k]}.*Game Timeout\|${DAYS[$i]}${HOURS[$k]}.*Game Reset" *audit* | sed 's/s//g' | awk '{sum += $4} END {print sum}')
- echo "${DAYS[$i]}${HOURS[$k]} Playtime $hourly_game_time" | tee -a "$HOURLY_FILE"
- done
- done
- #--Playtime Summary--
- PLAYTIME_FILE="$SUMMARY_DIR\\playtime.txt"
- echo "Generating File $PLAYTIME_FILE..."
- for (( i=0; i<${#LEVELS[@]}; i++ ))
- do
- # Playtime by Level
- plays=$(grep "${LEVELS[$i]}.*Level Start" *audit* | wc -l)
- playtime=$(grep "${LEVELS[$i]}.*Level" *audit* | sed 's/s//g' | awk '{sum += $5} END {print sum}')
- average_playtime=$(awk -v x="$playtime" -v y="$plays" 'BEGIN {print x / y}')
- echo "${LEVELS[$i]} Total Plays $plays" | tee -a $PLAYTIME_FILE
- echo "${LEVELS[$i]} Total Playtime $playtime" | tee -a $PLAYTIME_FILE
- echo "${LEVELS[$i]} Average Playtime $average_playtime" | tee -a $PLAYTIME_FILE
- done
- total_games=$(grep "Game Start" *audit* | wc -l)
- total_game_time=$(grep "Game End\|Game Timeout\|Game Reset" *audit* | sed 's/s//g' | awk '{sum += $4} END {print sum}')
- average_game_time=$(awk -v x="$total_game_time" -v y="$total_games" 'BEGIN {print x / y}')
- echo "Total Games $total_games" | tee -a $PLAYTIME_FILE
- echo "Total Game Time $total_game_time" | tee -a $PLAYTIME_FILE
- echo "Average Game Time $average_game_time" | tee -a $PLAYTIME_FILE
- mv $SUMMARY_DIR $USERPROFILE/Desktop
- echo "Analytic Summary Completed."
- sleep 10
The script running in action with data from MAGFest:
Take a minute or two to figure out what’s going on. Essentially we are looping through various arrays that contain the keywords found in our logs files, and use grep along with wc (word count) to tally up our totals. Then we output the totals to corresponding text files. Using this method we can tally up just about anything that we can key off of. So when we said that Logging is the backbone of our analytics, it really is! Without properly formatted log files this summary script would have been much harder to develop.
Utilizing the Analytics in Unity
Out of all the data we collected and summarized, what we find to be the most useful is the coordinate data. Meaning where the Player did the thing. While the totals and times are great for a quick glance as to how players are performing; the coordinate data is what gives us that extra bit of detail as to what exactly needs to be changed. Our Shell Script generates a coordinate file per level, and they look like this:
- SB r1 29.4,6.9
- SB r1 28.0,5.5
- SB r1 19.2,2.0
- SB r1 98.8,-18.2
- SB r1 28.1,4.3
- SB r1 27.5,6.3
- SB r1 28.3,6.7
- SB r1 28.3,6.7
- SB r1 49.3,-4.3
- SB r1 69.5,-4.8
- SB r1 29.2,6.9
- SB r1 33.9,5.3
- SB r1 49.0,-4.3
- SB r1 49.0,-1.9
- SB r1 55.1,-1.3
- SB r1 28.8,3.5
- SB r1 37.4,5.5
- SB r1 30.0,4.6
- SB r1 24.8,3.4
- SB r1 24.5,3.3
- SF r2 50.7,3.3
- SF r2 52.6,-2.9
- SF r2 132.9,2.3
- SF r2 53.0,0.0
- SF r2 51.4,-4.3
- SF r2 55.0,-3.6
- SF r2 48.2,1.8
- SF r2 49.6,1.6
- SF r2 49.6,-4.0
- SF r2 135.0,0.8
- SXB r1 95.2,-9.2
- SXB r2 55.5,-4.0
- SXB r2 54.8,-2.7
- SXB r2 85.4,-1.8
- SXB r2 48.5,1.7
- SXB r2 49.9,-1.4
- SXB r2 60.2,-6.1
- SXB r2 47.8,1.8
- SXB r2 47.8,1.8
- SXB r2 47.8,1.8
- SXB r2 47.8,1.8
- SXF r2 27.0,-4.6
- SXF r2 26.5,-1.3
- Glyph r1 12.4,1.6
- Glyph r1 12.6,2.1
- Glyph r1 12.6,2.5
- Glyph r1 12.2,1.6
- Glyph r1 12.5,2.0
- Glyph r1 12.3,1.6
- Glyph r1 12.4,1.0
- Glyph r1 12.6,2.4
- Glyph r1 12.7,2.6
- Glyph r1 12.5,2.2
- Glyph r1 12.4,2.4
- Glyph r1 12.7,2.5
- Glyph r1 12.4,2.3
- Glyph r1 12.5,2.1
- Glyph r1 12.6,2.3
- Glyph r1 12.3,2.0
- Glyph r1 12.4,2.4
- Glyph r1 12.5,2.1
- Glyph r1 12.3,2.0
- Glyph r1 12.5,2.7
- Glyph r1 12.8,2.1
- Glyph r1 12.4,2.2
- Glyph r1 12.4,2.2
- Glyph r1 12.4,1.9
- Glyph r1 12.3,1.8
- Glyph r1 12.5,2.3
- Glyph r1 12.8,2.6
- Glyph r1 12.4,2.1
- Glyph r1 12.3,2.2
- Glyph r2 26.9,-4.0
- Glyph r2 50.1,-1.8
- Glyph r2 49.4,-4.0
- Glyph r2 49.0,-4.0
- Glyph r2 49.1,-3.8
- Glyph r2 26.5,-4.0
- Glyph r2 49.1,-3.8
- Glyph r2 49.2,-1.2
- Glyph r2 49.6,-1.4
- Glyph r2 49.6,-2.3
- Glyph r2 48.7,-4.0
- Glyph r2 49.9,-2.2
- Glyph r2 48.9,-1.9
- Glyph r2 49.3,-2.2
- Glyph r2 49.6,-1.4
- Glyph r2 49.5,-1.7
- Glyph r2 49.7,-1.3
- Glyph r2 26.1,-4.6
- Glyph r2 49.4,-4.0
- Glyph r2 49.4,-4.0
- Glyph r2 49.6,-1.0
- Glyph r2 50.2,-3.4
- Glyph r2 48.7,-4.0
- Glyph r2 49.8,-4.0
- Death r1 12.5,-2.2
- Death r1 5.6,-2.4
- Death r1 42.5,-24.2
- Death r1 97.0,-24.1
- Death r1 15.8,-2.4
- Death r1 2.6,-1.9
- Death r1 73.9,-24.0
- Death r1 20.4,-1.4
- Death r1 1.9,-2.0
- Death r1 12.5,-2.2
- Death r1 4.5,-2.5
- Death r1 90.2,-23.9
- Death r1 7.4,-2.3
- Death r1 62.7,-24.2
- Death r1 19.8,-1.9
- Death r1 20.4,-1.4
- Death r1 2.6,-1.9
- Death r1 11.0,-2.3
- Death r1 14.7,-2.0
- Death r1 20.4,-1.4
- Death r1 15.8,-2.4
- Death r1 100.8,-23.9
- Death r1 12.5,-2.2
- Death r1 15.2,-2.0
- Death r1 12.5,-2.2
- Death r1 61.2,-24.1
- Death r1 5.8,-2.3
- Death r1 127.5,-24.2
- Death r1 12.5,-2.2
- Death r1 2.5,-1.9
- Death r1 20.4,-1.4
- Death r1 20.4,-1.4
- Death r1 20.4,-1.4
- Death r1 23.3,-1.0
- Death r1 155.9,-24.2
- Death r1 20.4,-1.4
- Death r1 21.4,-1.0
- Death r1 7.4,-2.3
- Death r1 7.4,-2.3
- Death r1 13.6,-2.4
- Death r1 141.1,-24.1
- Death r1 20.4,-1.4
- Death r1 20.4,-1.4
- Death r1 12.5,-2.2
- Death r1 12.5,-2.2
- Death r1 2.6,-1.9
- Death r1 7.4,-2.3
- Death r1 47.7,-24.2
- Death r1 12.5,-2.2
- Death r1 2.6,-1.9
- Death r1 22.6,-1.1
- Death r1 45.5,-24.0
- Death r1 20.4,-1.4
Then we bring all these coordinate files into Unity under a Resources -> Editor folder
Next we use our custom made “Heatmap” tool, that’s actually more of a Density Scatterplot, that will plot each piece of data for the corresponding level, parenting it to the appropriate row, and giving it the correct icon for the event.
In this Screenshot you can see how it’s definitely a Scatterplot and we call it a Density Scatterplot because we reduced the transparency for each icon. So they get brighter when there’s more stacked. This helps us detect the events of many as opposed to a stray event from a single person or two.
Notice this is plotting all the events for all our different Universes, which can get confusing quick. Luckily, our Script is setup to create the icons in a parenting structure that easily allows us to toggle the events we want to look at on/off.
Now we are just showing Deaths that occurred in the first universe aka r1. In our Hierarchy it looks like this:
It’s great reassurance when we come across an odd or challenging section of a level, and can toggle on the heatmap to see how other players have been performing in the same spot.
To make all this work and have the right icons go the right points, we developed two C# Scripts. The first is used to store the icon, row, and coordinate for each plot point, and the second is an EditorWindow that can load/unload the heatmap on the fly whether we’re in the editor or the game view:
- 1: HeatmapData.cs
- 2: HeatmapManager.cs
- using System;
- using UnityEngine;
- namespace Heatmap
- {
- public class HeatmapData
- {
- public HeatmapIcon Icon { get; private set; }
- public int Row { get; private set; }
- public Vector2 Coordinate { get; private set; }
- public HeatmapData(string data)
- {
- string[] fields = data.Split(' ');
- Icon = (HeatmapIcon) Enum.Parse(typeof(HeatmapIcon), "Heatmap_" + fields[0]);
- Row = int.Parse(fields[1].Remove(0, 1));
- string[] coordinates = fields[2].Split(',');
- Coordinate = new Vector2(float.Parse(coordinates[0]), float.Parse(coordinates[1]));
- }
- }
- }
This second script handles the bulk of the logic. Parsing the text files, creating the plot points, loading the correct icons, and then scattering the plot points across the level in an organized hierarchical fashion.
A lot of design, testing, and effort goes into making sure each level in ParaLily is an exciting, rewarding, and fair experience. With this method of logging everything/anything, summarizing it up, and bringing it all back in Unity; we can ensure our levels meet our high quality standards!
Thank you,
ParaLily Dev Team
Nate & Jeff
P.S. We skimmed over a lot of the technical setup and coding portions, so if you have any questions or comments feel free to email us info@ParaLilyGame.com and we'll get back to you when we can!