0010615: Game freezes after some time - new 0.44.06 optimizations?

(0037892)
Huntthetroll (reporter)
2018-03-12 15:47

No one else will be able to examine this bug unless you upload an archived copy of the relevant "region" folder to dffd.bay12games.com.

(0037893)
lethosor (manager)
2018-03-12 18:54
edited on: 2018-03-12 23:24

Also, this doesn't actually relate to artifacts, correct? Titles like that, while amusing, make searching/managing the bug tracker harder.

(0037896)
lethosor (manager)
2018-03-12 23:28

For reference, this was originally reported in https://www.reddit.com/r/dwarffortress/comments/83p9us/keep_losing_forts_to_game_freezing/ [^]

Another user in that thread reported seeing the same issue in an upgraded save, with 100% CPU usage. Does that also happen for you?

(0037907)
tkamat (reporter)
2018-03-13 06:18

I can confirm this issue, running Ubuntu 17.10. Considering everyone reporting this bug seems to run Linux, I suspect it is probably a problem with the Linux version of the game. My behavior is exactly the same as the OP, the game completely freezes seemingly randomly. I am running a brand new save on the vanilla DF.

(0037909)
lethosor (manager)
2018-03-13 07:38
edited on: 2018-03-13 07:39

tkamat, does your CPU usage from DF climb to 100%?

(0037913)
kneaugh (reporter)
2018-03-13 10:10

I am also having the same issue, CPU usage reaches 100%, the game stops responding, and the process has to be killed. Using Linux, vanilla DF, downloaded from the official source, no mods. In my case I don't know that uploading a save would be very instructive–every save I have has the issue eventually. Imported saves from older versions also have the same problem in 44.07 (I haven't tried it in 44.06).

There doesn't seem to be any predictable pattern to it, and sometimes the freeze happens during the initial two weeks of world activity before playing fort or adventure mode.

(0037915)
lethosor (manager)
2018-03-13 10:15

This was also reported in 0010619, with some more information there.

I suspect this is related to one of the optimizations Toady made, similar to 0008568 (for 0.40.17 on Windows).

(0037919)
life-saver (reporter)
2018-03-13 10:32
edited on: 2018-03-13 10:33

Since my issue is closed, I'll continue reporting here.

So, Issue still present in 44.07, same moment. Original init.txt & d_init.txt

I'll try under Windows next.

(0037922)
life-saver (reporter)
2018-03-13 11:18
edited on: 2018-03-13 11:39

Under Windows, it passed through the event.
I saved and brought it back to my Linux station.
But now I have to go back to work, so I'll test under linux tonight.

Consider this as proof that there really is something going on with the Linux version of DF since 44.06

At the rough moment of the event, I have in this save about 9 human adventurers visiting all at once. I'm still under siege.

(0037923)
life-saver (reporter)
2018-03-13 11:44

Logged back from work, DF process is frozen again.

Let me know if you need anything more. I'm glad to help in resolving this issue.

(0037933)
ab9rf (reporter)
2018-03-14 22:44
edited on: 2018-03-14 22:46

I'm fairly certain that this is a =/== mistake. Someone on IRC attached a debugger to a stuck game (64-bit linux, ASLR disabled) and determined that the game is stuck in a tight loop, traversing code from 0x000000000042c13f to 0x000000000042c200. The last three instructions in that range are a move of zero to a memory location, a test of that location against %rbp, and a jump not equal to the start of that range. %rbp is unchanged in that range, so this block of code, if entered with %rbp not equal to zero, will loop forever if none of the other tests lead it to escape.

This is almost certainly the result of writing an equality test (== 0) as an assignment (= 0), either in an if or, more likely, a for or while.

The relevant disassembly can be found at https://gist.github.com/ab9rf/dd79f770748112d08c4e3b4ad9ce407e [^]

(0037941)
Toady One (administrator)
2018-03-15 15:48

Hmm, I'd expect Windows freezes if it were that straightforward, but it could be combined in some horrible way with an uninitialized variable or an array bounds issue (the usual culprits on Linux-only bugs). Static analysis came up blank, so looks like I'm in for a longer 44.05/06 diff dive.

(0037967)
Toady One (administrator)
2018-03-18 02:54

Was hoping the diffs would show something before I left, but nothing popped out there either. Left with either output logs on a hopefully quick-freezing save, or figuring out how to analyze a freeze properly in linux internally w/ symbols etc. from my virtual setup, which hasn't been necessary in the past (virtual or otherwise) but might come up now.

(0037990)
Khym Chanur (reporter)
2018-03-21 19:03

You could try running a game under Valgrind, as Valgrind on my computer shows a bunch of "Conditional jump or move depends on uninitialised value(s)" errors, which might be the cause of the problem. That might help you quickly find where the problem is.

(0038029)
Loci (manager)
2018-03-25 19:00

v0.44.07: tristen posted a save in 0010663:

http://dffd.bay12games.com/file.php?id=13612 [^]

(0038039)
Toady One (administrator)
2018-03-26 20:34

I posted a test fix on the forum at http://www.bay12forums.com/smf/index.php?topic=169839.msg7720111#msg7720111 [^] .

The url is http://www.bay12games.com/dwarves/df_44_07_linux_test.tar.bz2 [^]

It worked on the freezing save I tested, but the whole thing is sort of dubious, though just the kind of array issue you might expect to optimize differently between operating systems. Or something. Valgrind stopped working before arriving there; not sure if that's a VM issue or what. So I found the problem code by (extensive) logging, though I'm not 100% certain it is fixed.

(0038047)
tkamat (reporter)
2018-03-27 14:14

I played for around a year on my freezing save, and the new patch seems to have fixed the problem. I haven't seen a single freeze or crash yet.

(0038048)
alpha (reporter)
2018-03-27 19:04

Loaded my reliably freezing save - no more freezes. The fix works.

Issue History
Date Modified	Username	Field	Change
2018-03-12 01:53	fuckbugs	New Issue
2018-03-12 15:47	Huntthetroll	Note Added: 0037892
2018-03-12 18:54	lethosor	Note Added: 0037893
2018-03-12 23:24	lethosor	Platform	Linux Version 44.06 => Linux
2018-03-12 23:24	lethosor	Summary	Urist McGame has created a masterful artifact *Total Game Crash* => Game freezes after some time
2018-03-12 23:24	lethosor	Description Updated	View Revisions
2018-03-12 23:24	lethosor	Note Edited: 0037893	View Revisions
2018-03-12 23:28	lethosor	Note Added: 0037896
2018-03-12 23:28	lethosor	Assigned To	=> lethosor
2018-03-12 23:28	lethosor	Status	new => needs feedback
2018-03-13 06:18	tkamat	Note Added: 0037907
2018-03-13 07:38	lethosor	Note Added: 0037909
2018-03-13 07:39	lethosor	Note Edited: 0037909	View Revisions
2018-03-13 10:10	kneaugh	Note Added: 0037913
2018-03-13 10:14	lethosor	Relationship added	has duplicate 0010619
2018-03-13 10:15	lethosor	Note Added: 0037915
2018-03-13 10:15	lethosor	Status	needs feedback => acknowledged
2018-03-13 10:15	lethosor	Summary	Game freezes after some time => Game freezes after some time - new 0.44.06 optimizations?
2018-03-13 10:15	lethosor	Tag Attached: 0.44.07
2018-03-13 10:32	life-saver	Note Added: 0037919
2018-03-13 10:33	life-saver	Note Edited: 0037919	View Revisions
2018-03-13 11:18	life-saver	Note Added: 0037922
2018-03-13 11:18	life-saver	Note Edited: 0037922	View Revisions
2018-03-13 11:21	life-saver	Note Edited: 0037922	View Revisions
2018-03-13 11:39	life-saver	Note Edited: 0037922	View Revisions
2018-03-13 11:44	life-saver	Note Added: 0037923
2018-03-14 22:44	ab9rf	Note Added: 0037933
2018-03-14 22:45	ab9rf	Note Edited: 0037933	View Revisions
2018-03-14 22:46	ab9rf	Note Edited: 0037933	View Revisions
2018-03-15 08:19	lethosor	Issue Monitored: Toady One
2018-03-15 08:19	lethosor	Severity	crash => block
2018-03-15 15:48	Toady One	Note Added: 0037941
2018-03-18 02:54	Toady One	Note Added: 0037967
2018-03-21 19:03	Khym Chanur	Note Added: 0037990
2018-03-21 19:03	Khym Chanur	Issue Monitored: Khym Chanur
2018-03-22 10:29	Huntthetroll	Issue Monitored: Huntthetroll
2018-03-23 02:18	alpha	Issue Monitored: alpha
2018-03-25 18:58	Loci	Relationship added	has duplicate 0010663
2018-03-25 19:00	Loci	Note Added: 0038029
2018-03-26 20:34	Toady One	Note Added: 0038039
2018-03-26 20:34	Toady One	Status	acknowledged => needs feedback
2018-03-27 14:14	tkamat	Note Added: 0038047
2018-03-27 19:04	alpha	Note Added: 0038048
2018-03-28 14:25	Toady One	Status	needs feedback => resolved
2018-03-28 14:25	Toady One	Fixed in Version	=> Next Version
2018-03-28 14:25	Toady One	Resolution	open => fixed
2018-03-28 14:25	Toady One	Assigned To	lethosor => Toady One
2018-04-02 20:12	Huntthetroll	Issue End Monitor: Huntthetroll

Notes
(0037892) Huntthetroll (reporter) 2018-03-12 15:47	No one else will be able to examine this bug unless you upload an archived copy of the relevant "region" folder to dffd.bay12games.com.

(0037893) lethosor (manager) 2018-03-12 18:54 edited on: 2018-03-12 23:24	Also, this doesn't actually relate to artifacts, correct? Titles like that, while amusing, make searching/managing the bug tracker harder.

(0037896) lethosor (manager) 2018-03-12 23:28	For reference, this was originally reported in https://www.reddit.com/r/dwarffortress/comments/83p9us/keep_losing_forts_to_game_freezing/ [^] Another user in that thread reported seeing the same issue in an upgraded save, with 100% CPU usage. Does that also happen for you?

(0037907) tkamat (reporter) 2018-03-13 06:18	I can confirm this issue, running Ubuntu 17.10. Considering everyone reporting this bug seems to run Linux, I suspect it is probably a problem with the Linux version of the game. My behavior is exactly the same as the OP, the game completely freezes seemingly randomly. I am running a brand new save on the vanilla DF.

(0037909) lethosor (manager) 2018-03-13 07:38 edited on: 2018-03-13 07:39	tkamat, does your CPU usage from DF climb to 100%?

(0037913) kneaugh (reporter) 2018-03-13 10:10	I am also having the same issue, CPU usage reaches 100%, the game stops responding, and the process has to be killed. Using Linux, vanilla DF, downloaded from the official source, no mods. In my case I don't know that uploading a save would be very instructive–every save I have has the issue eventually. Imported saves from older versions also have the same problem in 44.07 (I haven't tried it in 44.06). There doesn't seem to be any predictable pattern to it, and sometimes the freeze happens during the initial two weeks of world activity before playing fort or adventure mode.

(0037915) lethosor (manager) 2018-03-13 10:15	This was also reported in 0010619, with some more information there. I suspect this is related to one of the optimizations Toady made, similar to 0008568 (for 0.40.17 on Windows).

(0037919) life-saver (reporter) 2018-03-13 10:32 edited on: 2018-03-13 10:33	Since my issue is closed, I'll continue reporting here. So, Issue still present in 44.07, same moment. Original init.txt & d_init.txt I'll try under Windows next.

(0037922) life-saver (reporter) 2018-03-13 11:18 edited on: 2018-03-13 11:39	Under Windows, it passed through the event. I saved and brought it back to my Linux station. But now I have to go back to work, so I'll test under linux tonight. Consider this as proof that there really is something going on with the Linux version of DF since 44.06 At the rough moment of the event, I have in this save about 9 human adventurers visiting all at once. I'm still under siege.

(0037923) life-saver (reporter) 2018-03-13 11:44	Logged back from work, DF process is frozen again. Let me know if you need anything more. I'm glad to help in resolving this issue.

(0037933) ab9rf (reporter) 2018-03-14 22:44 edited on: 2018-03-14 22:46	I'm fairly certain that this is a =/== mistake. Someone on IRC attached a debugger to a stuck game (64-bit linux, ASLR disabled) and determined that the game is stuck in a tight loop, traversing code from 0x000000000042c13f to 0x000000000042c200. The last three instructions in that range are a move of zero to a memory location, a test of that location against %rbp, and a jump not equal to the start of that range. %rbp is unchanged in that range, so this block of code, if entered with %rbp not equal to zero, will loop forever if none of the other tests lead it to escape. This is almost certainly the result of writing an equality test (== 0) as an assignment (= 0), either in an if or, more likely, a for or while. The relevant disassembly can be found at https://gist.github.com/ab9rf/dd79f770748112d08c4e3b4ad9ce407e [^]

(0037941) Toady One (administrator) 2018-03-15 15:48	Hmm, I'd expect Windows freezes if it were that straightforward, but it could be combined in some horrible way with an uninitialized variable or an array bounds issue (the usual culprits on Linux-only bugs). Static analysis came up blank, so looks like I'm in for a longer 44.05/06 diff dive.

(0037967) Toady One (administrator) 2018-03-18 02:54	Was hoping the diffs would show something before I left, but nothing popped out there either. Left with either output logs on a hopefully quick-freezing save, or figuring out how to analyze a freeze properly in linux internally w/ symbols etc. from my virtual setup, which hasn't been necessary in the past (virtual or otherwise) but might come up now.

(0037990) Khym Chanur (reporter) 2018-03-21 19:03	You could try running a game under Valgrind, as Valgrind on my computer shows a bunch of "Conditional jump or move depends on uninitialised value(s)" errors, which might be the cause of the problem. That might help you quickly find where the problem is.

(0038029) Loci (manager) 2018-03-25 19:00	v0.44.07: tristen posted a save in 0010663: http://dffd.bay12games.com/file.php?id=13612 [^]

(0038039) Toady One (administrator) 2018-03-26 20:34	I posted a test fix on the forum at http://www.bay12forums.com/smf/index.php?topic=169839.msg7720111#msg7720111 [^] . The url is http://www.bay12games.com/dwarves/df_44_07_linux_test.tar.bz2 [^] It worked on the freezing save I tested, but the whole thing is sort of dubious, though just the kind of array issue you might expect to optimize differently between operating systems. Or something. Valgrind stopped working before arriving there; not sure if that's a VM issue or what. So I found the problem code by (extensive) logging, though I'm not 100% certain it is fixed.

(0038047) tkamat (reporter) 2018-03-27 14:14	I played for around a year on my freezing save, and the new patch seems to have fixed the problem. I haven't seen a single freeze or crash yet.

(0038048) alpha (reporter) 2018-03-27 19:04	Loaded my reliably freezing save - no more freezes. The fix works.