Dwarf Fortress Bug Tracker - Dwarf Fortress
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0005986Dwarf FortressDwarf Mode -- Interface, Designationspublic2012-06-04 03:262013-12-07 04:30
ReporterLeonassan 
Assigned To 
PriorityurgentSeveritymajorReproducibilityalways
StatusnewResolutionopen 
PlatformIBM CompatibleOSWindows 7 64 SP1OS Version6.1.7601
Product Version0.34.10 
Target VersionFixed in Version 
Summary0005986: Designating large areas for smoothing slows game to a crawl
DescriptionWhenever I designate large areas of the fortress for smoothing, the game drops to single digit FPS. The larger the area, the higher the impact, and it gradually recovers as the area to be smoothed gets smaller.
Steps To ReproduceSelect large area to smooth.
Additional InformationThis seems to have gotten worse as I've moved my fort deeper underground. Current area that is being smoothed is at level 44.
TagsNo tags attached.
Relationships
Attached Files
Issue History
Date ModifiedUsernameFieldChange
2012-06-04 03:26LeonassanNew Issue
2012-06-04 03:26LeonassanIssue Monitored: Leonassan
2012-06-04 03:26LeonassanIssue End Monitor: Leonassan
2012-06-04 08:10FootkerchiefNote Added: 0022846
2012-06-04 13:39LeonassanNote Added: 0022849
2012-06-04 14:01LeonassanNote Added: 0022850
2012-06-05 03:21MopsyNote Added: 0022856
2012-06-05 03:29MopsyIssue Monitored: Mopsy
2012-06-05 05:32blake77Note Added: 0022858
2012-06-05 11:34agNote Added: 0022859
2012-06-05 17:31LeonassanNote Added: 0022865
2012-06-05 17:32LeonassanNote Edited: 0022865bug_revision_view_page.php?bugnote_id=0022865#r8505
2012-06-05 17:34LeonassanNote Edited: 0022865bug_revision_view_page.php?bugnote_id=0022865#r8506
2012-06-05 18:13QuietustNote Added: 0022867
2012-06-05 18:17LeonassanNote Added: 0022868
2012-06-06 02:44agNote Added: 0022870
2012-06-06 03:20agNote Edited: 0022870bug_revision_view_page.php?bugnote_id=0022870#r8508
2012-06-06 12:54LeonassanNote Added: 0022877
2012-06-07 01:45BuglistIssue Monitored: Buglist
2013-12-07 04:30ThundercraftIssue Monitored: Thundercraft

Notes
(0022846)
Footkerchief   
2012-06-04 08:10   
Reminder sent to: Leonassan

Does this reproduce in every fortress? If not, please upload a save demonstrating the problem to http://dffd.wimbli.com/ [^] and post the link here.
(0022849)
Leonassan   
2012-06-04 13:39   
Done. I haven't managed to get a second fort deep enough to replicate, so here's the save. http://dffd.wimbli.com/file.php?id=6423 [^]
(0022850)
Leonassan   
2012-06-04 14:01   
It's very apparent where the save currently is. I loaded it up and was at 7FPS, as soon as the dwarves finished their current engrave designation, it jumped to 31.
(0022856)
Mopsy   
2012-06-05 03:21   
I've been noticing similar behavior in a large, populous 40d fort. Could it be a consequence of the alleged "dwarves don't seek jobs, jobs seek dwarves" design? Unlike a designation for mining into solid rock, a smoothing designation often creates a large number of simultaneously reachable tasks. If all those tasks go looking for an engraver once every N steps, we might see FPS drop significantly. The FPS drop might also depend on the number of dwarves in the fort: (CPU cycles used for smoothing job assignment) = K * n(reachable smoothing-designated tiles) * n(dwarves to check for availability as engravers).
(0022858)
blake77   
2012-06-05 05:32   
I also noticed this behaviour in previous versions of DF.
(0022859)
ag   
2012-06-05 11:34   
Ran this through my profiling setup on Core i5 (got 20 FPS):

http://cloud.github.com/downloads/angavrilov/dfprofile/log.slow-engrave.4a.svg [^]

Observations:
1) Engraving job creation does take more than 7% CPU; 5 of those is in the loop over all units, including dead. I would guess the function is called for every designated tile.
2) More than 15% CPU is used by searching for items by id - general_ref_item::getItem is obvious, but there are also other instances inlined in various other functions, e.g. 2% in map_block::updateTemp.
3) There are also other item-related functions mentioned, including things like the map renderer, which walks through all items on the map to check if they are visible now.

This supports the generally accepted opinion that large number of units (2500+ on this map) and items (50000+) causes lag.

Note: in viewing the graph bear in mind that true stack traces are unavailable, so the 'total time spent' percentage in parentheses is computed purely heuristically, and may significantly deviate from the truth.
(0022865)
Leonassan   
2012-06-05 17:31   
(edited on: 2012-06-05 17:34)
Perhaps pruning dead units from the units list will alleviate the problem. I'll try it and see if it helps any. I didn't notice the slowdown until after I breached the caverns, so that could be where the additional units are coming from. Perhaps the loop to check for units for a job should be limited to alive dwarves only?
Edit:
Also, to check the theory of large multiple accessible jobs, I've found the same behavior with mining designations that can all be reached, and massive dumping designations, so I believe you are correct that it has to do with a lot of tasks that can be completed being queued at the same time.
Edit:
Ok, I cannot remove dead units from the units list.
(0022867)
Quietust   
2012-06-05 18:13   
> Ok, I cannot remove dead units from the units list.

Of course you can't - you'd need DFHack to do something like that.
(0022868)
Leonassan   
2012-06-05 18:17   
> Of course you can't - you'd need DFHack to do something like that.

Or since you can remove units from your alive, active dwarves list, this ability should be added to the Dead/Missing units list as well.
(0022870)
ag   
2012-06-06 02:44   
(edited on: 2012-06-06 03:20)
Some experimental results:
- Nuking all items except those in buildings and constructions got fps from 20 to 50.
- Further 5 fps recovered by sealing that pump contraption - you should never make areas with changing water level reachable from the rest of the fortress.
- What remained is 20% unit temperature update cost, 20% unit enemy/friend checks, 20% pathfinding through all those long and wide areas as units drag plants to stockpiles.

Edit: here's the graph: http://cloud.github.com/downloads/angavrilov/dfprofile/log.slow-engrave.5d.svg [^]
(0022877)
Leonassan   
2012-06-06 12:54   
I'm sure there are other things that are causing lag, but that doesn't change the fact that smoothing is causing lag :)