Dwarf Fortress Bug Tracker - Dwarf Fortress
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0010016Dwarf FortressWorld Generation -- Generalpublic2016-09-25 03:302016-10-09 19:33
ReporterEsavier 
Assigned ToToady One 
PrioritynormalSeveritycrashReproducibilityalways
StatusresolvedResolutionfixed 
PlatformWindowsOSWindows 10OS Version14393.187
Product Version0.43.05 
Target VersionFixed in Version0.43.05 
Summary0010016: Out of memory crash
DescriptionOn modded Dwarf fortress (Masterwork 1.16), but propably also on stock.
During world generation, with large nubmber of civs, megabeasts, beasts, and titans, DF advanced worldgen will crash in the same place.
After patching binary with LargeAddressAware, problem is resolved.
Steps To Reproduce1.Advanced World Generation, set:
Large or Medium Region
max-subregions:5000
Titans:1000
Megabeats:1000
Semi-Megabeasts:1000
Civilizations:55
2. generate
crash is expected between histroy generation and placing civilizations
Additional InformationLarge Address Aware link:
https://www.techpowerup.com/forums/threads/large-address-aware.112556/ [^]

No error logs anywhere.
Additionally patched: dfhack and binpatch
TagsNo tags attached.
Relationships
Attached Files
Issue History
Date ModifiedUsernameFieldChange
2016-09-25 03:30EsavierNew Issue
2016-09-25 10:56lethosorNote Added: 0035897
2016-09-25 10:56lethosorAssigned To => lethosor
2016-09-25 10:56lethosorStatusnew => resolved
2016-09-25 10:56lethosorResolutionopen => not fixable
2016-09-25 10:56lethosorProduct Version0.43.05 => 0.43.03
2016-09-25 10:56lethosorFixed in Version => 0.43.05
2016-09-27 04:48EsavierNote Added: 0035912
2016-09-27 04:48EsavierStatusresolved => needs feedback
2016-09-27 04:48EsavierResolutionnot fixable => reopened
2016-09-27 04:48EsavierNote Edited: 0035912bug_revision_view_page.php?bugnote_id=0035912#r14481
2016-09-27 07:43LociNote Added: 0035915
2016-09-27 12:54EsavierNote Added: 0035918
2016-09-27 12:54EsavierStatusneeds feedback => assigned
2016-09-30 16:27lethosorNote Added: 0035921
2016-09-30 16:27lethosorStatusassigned => needs feedback
2016-09-30 16:27lethosorProduct Version0.43.03 => 0.43.05
2016-10-06 09:35EsavierNote Added: 0035950
2016-10-06 09:35EsavierStatusneeds feedback => assigned
2016-10-09 19:33lethosorStatusassigned => resolved
2016-10-09 19:33lethosorResolutionreopened => fixed
2016-10-09 19:33lethosorAssigned Tolethosor => Toady One

Notes
(0035897)
lethosor   
2016-09-25 10:56   
MDF 1.16 uses DF 0.43.03, not 0.43.05: http://www.bay12forums.com/smf/index.php?topic=160759.0 [^]

There's really nothing that can be done here, short of generating a smaller world. Upgrading to 0.43.05 should fix the issue, so I would bring that up to Meph if nobody has yet.
(0035912)
Esavier   
2016-09-27 04:48   
I am not playing on stock MDF, i fixed it up to work with newest DF, however lot features are disabled. I double checked, that this issue is not corelated to modifications made to MDF.

Issue persist on stock DF. I tested it through latest two straight days and i narrowed the issue to memory limit problem, without patch its almost always near 2GB, with patch it will crash near 3.8 GB, 4GB including stale memory (paging).

I also confirmed with microsoft documentation that x86 application (32bit) are by design limited to roughly 4GB.

I belive that this is time to DF to catch up to modern machines, less than 5% devices capable of running DF are not able to use 64bit architectures, and almost always there is present more than 4GB of physical ram - standard was 8GB about 4 years ago for energy saving laptops.

Limiting memory will also hamper community's ability to mod and adjust DF, and as such, user experience. Leaving this issue open will either way pop up in future, together with multithreading, because in some point, complexity of computations will take too much toll on users, limiting FPS to unplayable values.

Relatively easy fix is to start making and/or change current build to 64bit architecture. This should also help with performance issues.
(0035915)
Loci   
2016-09-27 07:43   
Dwarf Fortress switched to a 64-bit build in version 0.43.05, as mentioned in the development log:

http://www.bay12games.com/dwarves/ [^]


Does vanilla 0.43.05 crash in the manner you stated?
(0035918)
Esavier   
2016-09-27 12:54   
yes indeed. I am sure that i set up properly 43.05 on top of some of Masterwork Raws that do not use special features provided by dfhack. Game is working properly on v.small-small maps for more than 10 in game years.

This may be corelated with fact that i am using Insider Preview version of Windows 10, but i seriously doubt it. I will try to set up clean stable windows and test it again but it will take some time (till this sunday for sure)
(0035921)
lethosor   
2016-09-30 16:27   
Are you using the 32-bit or 64-bit build of 0.43.05?

(Also, I'd like to point out that 64-bit support wasn't simple, and it should not have a very large impact on speed.)
(0035950)
Esavier   
2016-10-06 09:35   
ok, on freshly installed windows 10 stable i couldn't reproduce this bug. Propably something related to windows insider preview release, or just virtualbox lies to me for some reason.

i used x64 build from DFFD.

From my experience migration to wider architecture is not particulary hard. Also from my experience and valgrind stats, difference on this same code between x64 an x86 is on avg. 4-20% faster (depends on code). Notice that i wrote "relatively", and "should" - those indicate that it is statement taken upon my knowledge as programmer who, however, is be not familiar with code but familiar with language.

Still i am glad that it is building on x64 already. good work:)

Please close it for now. If i came up with something i will reopen.