Dwarf Fortress Bug Tracker - Dwarf Fortress
View Issue Details
IDProjectCategoryView StatusDate SubmittedLast Update
0003942Dwarf FortressDwarf Mode -- Economicspublic2011-02-10 15:532013-06-07 18:02
ReporterTer13 
Assigned ToToady One 
PriorityurgentSeveritymajorReproducibilityalways
StatusresolvedResolutionfixed 
PlatformOSOS Version
Product Version0.31.18 
Target VersionFixed in Version0.34.06 
Summary0003942: Dwarves owning broken clothing. Clothes don't rot, ANNIHILATES FPS.
DescriptionThere is something seriously wrong with the way Dwarves handle clothing ownership. My FPS, and the FPS of a friend have suffered continually. If our fortress has 200 dwarves, we manage 100 FPS, but if those dwarves wear clothing (we turned it off and tried it), the FPS steadily decreases as more and more dwarves wear out their clothing.

It seems that Dwarves are constantly checking to see if they need new clothes once they own a broken piece of clothing, and this is causing excess processing to be done each frame.
Steps To ReproduceStart fortress with dwarves with clothing. increase population to 200 before clothes break. Good FPS.

Wait until clothes break. Shit FPS.


Start another fortress with the dwarven civ not wearing clothes. Good FPS no matter what.
Additional InformationTurn off the wear clothing behavior in the civ. This will clearly show that there is something very, very wrong with item ownership.
TagsNo tags attached.
Relationships
related to 0002481resolved Toady One Dwarves will claim new clothes when theirs is worn (xClothesx) but not wear it 
related to 0004207resolved Toady One Inescapable, inevitable, intolerable lag 
Attached Files
Issue History
Date ModifiedUsernameFieldChange
2011-02-10 15:53Ter13New Issue
2011-02-11 08:38FootkerchiefNote Added: 0015082
2011-02-11 08:38FootkerchiefTag Attached: AWAITING UPDATE
2011-02-11 09:46Ter13Note Added: 0015086
2011-02-11 10:05Ter13Note Edited: 0015086bug_revision_view_page.php?bugnote_id=0015086#r5706
2011-02-11 15:29Hieronymous AlloyIssue Monitored: Hieronymous Alloy
2011-02-14 15:10KogutNote Added: 0015104
2011-02-14 15:10KogutIssue Monitored: Kogut
2011-02-14 15:40IT 000Note Added: 0015105
2011-02-15 02:34KogutNote Added: 0015107
2011-02-16 09:35NarosIssue Monitored: Naros
2011-02-16 11:17hyndisNote Added: 0015122
2011-03-16 12:06FootkerchiefRelationship addedrelated to 0004207
2011-03-16 15:32Malibu StaceyIssue Monitored: Malibu Stacey
2011-03-16 18:19FootkerchiefNote Added: 0016309
2011-03-17 01:32AbuDhabiNote Added: 0016318
2011-03-17 01:32AbuDhabiIssue Monitored: AbuDhabi
2011-03-17 13:36AbuDhabiNote Edited: 0016318bug_revision_view_page.php?bugnote_id=0016318#r6106
2011-03-30 14:21Orkel2Note Added: 0016816
2011-03-30 14:21Orkel2Note Edited: 0016816bug_revision_view_page.php?bugnote_id=0016816#r6262
2011-03-30 14:39FootkerchiefIssue Monitored: Orkel2
2011-03-30 14:39FootkerchiefNote Added: 0016818
2011-03-31 00:16profitNote Added: 0016827
2011-03-31 00:39profitNote Added: 0016830
2011-03-31 00:49InsanityPreludeNote Added: 0016831
2011-03-31 00:49InsanityPreludeIssue Monitored: InsanityPrelude
2011-03-31 01:03AbuDhabiNote Added: 0016832
2011-03-31 08:01KogutNote Added: 0016850
2011-03-31 09:21InsanityPreludeNote Added: 0016858
2011-03-31 09:32FootkerchiefRelationship addedrelated to 0002481
2011-04-09 08:40DekonNote Added: 0017232
2011-04-09 17:39alcohol_dependentNote Added: 0017259
2011-04-09 17:41alcohol_dependentNote Edited: 0017259bug_revision_view_page.php?bugnote_id=0017259#r6421
2011-04-09 18:22alcohol_dependentNote Edited: 0017259bug_revision_view_page.php?bugnote_id=0017259#r6422
2011-04-09 19:26alcohol_dependentNote Edited: 0017259bug_revision_view_page.php?bugnote_id=0017259#r6423
2011-04-10 10:31alcohol_dependentNote Edited: 0017259bug_revision_view_page.php?bugnote_id=0017259#r6424
2011-04-13 09:20kwielandNote Added: 0017337
2011-04-13 14:25FootkerchiefNote Added: 0017347
2011-04-18 17:22devekNote Added: 0017434
2011-04-19 04:09agNote Added: 0017441
2011-04-20 20:51DekonNote Added: 0017452
2011-05-01 02:41agNote Added: 0017591
2011-07-09 03:55BeeskeeIssue Monitored: Beeskee
2011-09-14 14:15eastcIssue Monitored: eastc
2011-09-15 23:25gmafkNote Added: 0018727
2011-09-16 14:06MorrolanIssue Monitored: Morrolan
2011-09-19 14:55Xen0nNote Added: 0018742
2011-09-20 18:45gmafkNote Added: 0018745
2011-09-20 20:07kwielandNote Added: 0018746
2011-09-20 20:09kwielandNote Edited: 0018746bug_revision_view_page.php?bugnote_id=0018746#r7041
2011-09-20 20:10kwielandNote Edited: 0018746bug_revision_view_page.php?bugnote_id=0018746#r7042
2012-02-26 11:46BuglistIssue Monitored: Buglist
2012-02-29 01:25kentoIssue Monitored: kento
2012-03-15 12:13thvazNote Added: 0021488
2012-03-18 00:24thvazNote Edited: 0021488bug_revision_view_page.php?bugnote_id=0021488#r8046
2012-03-18 07:23DwarfuTag Detached: AWAITING UPDATE
2012-03-18 07:24DwarfuIssue Monitored: Toady One
2012-03-18 07:56KogutNote Added: 0021554
2012-03-19 04:01Toady OneNote Added: 0021588
2012-03-19 04:01Toady OneStatusnew => resolved
2012-03-19 04:01Toady OneFixed in Version => Next Version
2012-03-19 04:01Toady OneResolutionopen => fixed
2012-03-19 04:01Toady OneAssigned To => Toady One
2012-03-21 15:07KogutIssue End Monitor: Kogut
2012-03-23 05:00BuglistIssue End Monitor: Buglist
2012-03-29 05:39NarosIssue End Monitor: Naros
2012-03-30 10:21AbuDhabiIssue End Monitor: AbuDhabi
2012-05-28 17:53MorrolanIssue End Monitor: Morrolan
2012-07-05 06:46Malibu StaceyIssue End Monitor: Malibu Stacey
2013-06-07 18:02InsanityPreludeIssue End Monitor: InsanityPrelude

Notes
(0015082)
Footkerchief   
2011-02-11 08:38   
It would be helpful to post saves that demonstrate this behavior. http://dffd.wimbli.com/ [^]

Overall object count is suspected of causing the biggest FPS hit. Are you able to exclude this factor? I.e., when you have have two fortresses, one with clothes and one without, are the item counts still on par?
(0015086)
Ter13   
2011-02-11 09:46   
(edited on: 2011-02-11 10:05)
The idea that item count affects FPS is indicative of overall design issues. Items shouldn't be processed every frame unless there is behavior connected to those items.

The item counts should have been roughly similar, as without clothes, I was able to carry on much longer, and as such produce a much larger quantity of items, all the while my FPS remained stable.

I am very certain that there is a bit of code running unintentionally due to the owned and broken clothing. I believe the dwarf is constantly seeking the nearest replacement for that broken clothing, even though he has a second bit of clothing already. I believe that this is a leading cause to FPS hits, and should be pretty easy to profile and confirm.
(0015104)
Kogut   
2011-02-14 15:10   
How can I disable clothes?
(0015105)
IT 000   
2011-02-14 15:40   
remove [CLOTHING] and [SUBTERRANEAN_CLOTHING] from the dwarven entity file.
(0015107)
Kogut   
2011-02-15 02:34   
I did this, but on embark dwarfs are still wearing sth. Is it supposed to happen?
(0015122)
hyndis   
2011-02-16 11:17   
I just delete clothing from both the entity files and also in the raws. I leave the armor but clothes are deleted outright. Works fine.
(0016309)
Footkerchief   
2011-03-16 18:19   
Reminder sent to: Ter13

We need a save for this report. Please upload one to http://dffd.wimbli.com/ [^]
(0016318)
AbuDhabi   
2011-03-17 01:32   
(edited on: 2011-03-17 13:36)
Do you think that modifying clothes to have the [HARD] token solve this problem? This would prevent them from rotting at all, like armour does.

EDIT: It seems that just modding out [SOFT] isn't sufficient to battle wear and tear. Though much slower, the clothes do get used up.
(0016816)
Orkel2   
2011-03-30 14:21   
This is happening to me too.

Good clothes, ~ 100 FPS on 120 dwarves

Clothes starting to rot, FPS has steadily dropped to 25 in the past year or so.
(0016818)
Footkerchief   
2011-03-30 14:39   
Reminder sent to: Orkel2

Please upload a save demonstrating lag from rotting clothes. Bonus points if you can also provide a save from before the clothes start rotting. http://dffd.wimbli.com/ [^]
(0016827)
profit   
2011-03-31 00:16   
I too would really like to see this... I would like to test and see if removing their clothing via putting them into the military with blank uniforms that replace clothing would fix it as well.
(0016830)
profit   
2011-03-31 00:39   
I just tested this on one of my forts, using the military feature I was able to get everyone out of their clothes... (In life I usually just have to ask and they drop) Many of these clothes were completely deteriorated, but I did not experience a FPS gain after I returned the dorfs to civ state even though they did not put on new clothing. ( I forbade it) I am unsure where the FPS boost is coming from but I have a sneaking suspicion it has something to do with the rot code or contaminants not the dorfs trying to replace them.
(0016831)
InsanityPrelude   
2011-03-31 00:49   
Does anything happen if you destroy the rotted clothes? (e.g. throw into magma)
(0016832)
AbuDhabi   
2011-03-31 01:03   
Destroying them is difficult, because they are owned, and therefore can't be dumped.
(0016850)
Kogut   
2011-03-31 08:01   
child of 0002481?
(0016858)
InsanityPrelude   
2011-03-31 09:21   
Oh, right. I forgot about that.
(0017232)
Dekon   
2011-04-09 08:40   
Seems common to me as well. My FPS plummets after any invasion attempt until I can manage to clean up all their crap and put it through an atom smasher. Which means my dwarves end up spending a ludicrous amount of each month just playing clean up from the last invasion.
(0017259)
alcohol_dependent   
2011-04-09 17:39   
(edited on: 2011-04-10 10:31)
I am going to try removing clothes as well and will post my results.. I have a 3 ghz processor, typically get a big slowdown around 80 dwarves. Never go much beyond there but if I leave temp on with a pop cap at 80 my FPS will slow to 50 with a moderately inefficient fort (35-40 with a lot of designations, or if I decide to style a very inefficient fort).

Will update with results of my next fort. For now I guess, I'll just forge a pair of boots for every-dwarf to sidestep the potential infection problem.


*Update*

Doesn't seem to have made any significant FPS improvements. Much less clutter though. Will tinker more.
(0017337)
kwieland   
2011-04-13 09:20   
I have an established fort (15+ years) 60 dwarfs, half kids. 3x3 embark area. No original clothes left, they've all rotted away. However, all my dwarfs are equiped with cloth cloaks (6), trousers, and socks. These items routinely wear. My normal is 150 FPS. All my dwarfs have rooms with chests and cabinets. Do yours?

My FPS goes down to 40-50 after a siege when I've got a lot of crap to pick up from ground level. I think the FPS hit is due to lots of pathing calculations. Somewhere I read that the path calculation is redone everytime a dwarf runs into another dwarf. Also, during "normal" times, lots of the dwarfs are constructing things, which doesn't call pathing.

My take: my FPS hit is due to pathing not wear.
(0017347)
Footkerchief   
2011-04-13 14:25   
Reminder sent to: Orkel2, Ter13

This report will be resolved as "unable to reproduce" in the near future unless saves are uploaded that demonstrate that rotting clothes have a significant impact on FPS, independent of other factors.
(0017434)
devek   
2011-04-18 17:22   
@Ter13

>The idea that item count affects FPS is indicative of overall design issues.

>I am very certain that there is a bit of code running unintentionally due to the
>owned and broken clothing. I believe the dwarf is constantly seeking the nearest
>replacement for that broken clothing, even though he has a second bit of
>clothing already. I believe that this is a leading cause to FPS hits, and should
>be pretty easy to profile and confirm.

How can it be profiled and confirmed if no one ever bothers to upload relevant saves? I think that is more of a design issue with the user base :P

You don't even need Toady to do it, I can use memory hacking to unown all items and then delete them manually to see... if I had a save to work with that is :)
(0017441)
ag   
2011-04-19 04:09   
I tried unowning, dumping into a smasher and pulling the lever on all worn clothes with my new cleanowned tool in DFHack, and it gave at most 10-15% improvement on my save (quite near to the statistical noise level).

Dumping all the cheap junk my stonecrafter created over the years onto a caravan seemed to have a similar effect. The latter was actually inspired by some specific results oprofile gave me, namely by relatively large time spent in binary search to find the container item of an object.
(0017452)
Dekon   
2011-04-20 20:51   
Maybe in a few weeks once classes are all done I'll try and do a fort and collect a crap ton of goblin junk and have it set up for a save, so at least leave this open for a bit longer.

If I've heard correctly (and I may not be correct about this), items have to constantly check themselves to determine 'Am I muddy? Am I on fire? Am I needing to be moved to a stockpile/dump? Am I...' etc etc. Because they're constantly being checked for this, it creates an intense drop in FPS when items start to build dramatically simply because so many things are undergoing this process of self check so frequently.
(0017591)
ag   
2011-05-01 02:41   
After some more profiling and staring at assembly, I have an easy optimization suggestion: reimplement flagarrayst::has_flag as a self-contained inline function without any external calls. It is used a lot for quering flags in raw structures, and currently every usage results in a cross-dll call without any opportunity for the compiler to use the fact that the flag ID is known at compile time. In the linux version get_slot_and_addbit_uchar accounts for 50% of all time spent in libgraphics.so.
(0018727)
gmafk   
2011-09-15 23:25   
I think I have proof that clothing is a source of a significant drop in FPS. My fort (31.25, linux) started at 100 FPS and every migrant wave reduced it until it was running at 30-40 FPS with 120 dwarves.

Save: http://dffd.wimbli.com/file.php?id=4943 [^]

All 120 dwarves are gathered near the garbage dump (F5) and are idle. All armor, legwear, headwear, handwear and footwear is designated for dumping (but dwarves own it so they won't dump it).

Step 1: Do nothing for 10-20s and let the FPS counter stabilize (39 FPS)
Step 2: Run 'cleanowned all' in DFHack
Step 3: Watch your FPS rise as dwarves discard their clothing into lava (54 FPS). This step can take around 2 minutes - you can use the stocks menu to check how many items have been destroyed so far).

Improvement from 39 FPS (25.64 ms) to 54 FPS (18.51 ms) is pretty significant: 7.12 ms spent per frame just to process clothes.
(0018742)
Xen0n   
2011-09-19 14:55   
Perhaps, add a control in case it was just getting rid of so many items that gave the improvement, not necessarily clothes. Take the same save and dump an equal number of crafts (or rocks or something) and compare the FPS improvement to when you dumped all the clothes.
(0018745)
gmafk   
2011-09-20 18:45   
I used the save from my previous post: http://dffd.wimbli.com/file.php?id=4943 [^]

Items other than clothes can be easily tested using 'autodump destroy'. Unfortunately, it doesn't seem to work on equipped items, so I just copied results from my previous test for comparison.

Step 1: Mark items for dumping.
Step 2: Run 'autodump destroy' in DFHack.

The game starts at 39 FPS (25.64 ms) before each test. Results:

Destroying 1,7 k clothes: 54 FPS (18.51 ms), -7.12 ms
Destroying 1,7 k stone (limestone + gabbro): 40 FPS (25.00 ms), -0.64 ms
Destroying 4,5 k stone (all stone on the map): 42 FPS (23.80 ms), -1.83 ms

It seems that processing clothes takes a lot more time than processing rocks.
(0018746)
kwieland   
2011-09-20 20:07   
(edited on: 2011-09-20 20:10)
gmafk, good work. I think the clothes are PART of the issue, but I think there are other things as well (like contaminants, pathing). I was going to suggest abandoning and reclaiming, but there are other bugs there, so it would be hard to isolate the real cause.

The original bug report states:"Dwarves are constantly checking to see if they need new clothes once they own a broken piece of clothing, and this is causing excess processing to be done" Do you think this is the case? Wouldn't that make your FPS drop after you "liberate" the clothes? Have you run the liberated game for a year to see if the FPS stays up?
(0021488)
thvaz   
2012-03-15 12:13   
(edited on: 2012-03-18 00:24)
This one shouldn't be fixed by today's devlog?
(0021554)
Kogut   
2012-03-18 07:56   
It is hidden (workaround is to simply produce clothes) but probably if "dwarves wear clothing (we turned it off and tried it), the FPS steadily decreases as more and more dwarves wear out their clothing".
(0021588)
Toady One   
2012-03-19 04:01   
As far as I can tell from having a few hundred dwarves around and rotting their clothing and placing clothing piles and so on, this overall situation is a lot better. That's not to say there aren't going to be problems, especially the ongoing general issues from having lots of objects around, but I'm going to mark this one off.