Mantis Bug Tracker

View Issue Details Jump to Notes ] Issue History ] Print ]
IDProjectCategoryView StatusDate SubmittedLast Update
0003942Dwarf FortressDwarf Mode -- Economicspublic2011-02-10 15:532013-06-07 18:02
ReporterTer13 
Assigned ToToady One 
PriorityurgentSeveritymajorReproducibilityalways
StatusresolvedResolutionfixed 
PlatformOSOS Version
Product Version0.31.18 
Target VersionFixed in Version0.34.06 
Summary0003942: Dwarves owning broken clothing. Clothes don't rot, ANNIHILATES FPS.
DescriptionThere is something seriously wrong with the way Dwarves handle clothing ownership. My FPS, and the FPS of a friend have suffered continually. If our fortress has 200 dwarves, we manage 100 FPS, but if those dwarves wear clothing (we turned it off and tried it), the FPS steadily decreases as more and more dwarves wear out their clothing.

It seems that Dwarves are constantly checking to see if they need new clothes once they own a broken piece of clothing, and this is causing excess processing to be done each frame.
Steps To ReproduceStart fortress with dwarves with clothing. increase population to 200 before clothes break. Good FPS.

Wait until clothes break. Shit FPS.


Start another fortress with the dwarven civ not wearing clothes. Good FPS no matter what.
Additional InformationTurn off the wear clothing behavior in the civ. This will clearly show that there is something very, very wrong with item ownership.
TagsNo tags attached.
Attached Files

- Relationships
related to 0002481resolvedToady One Dwarves will claim new clothes when theirs is worn (xClothesx) but not wear it 
related to 0004207resolvedToady One Inescapable, inevitable, intolerable lag 

-  Notes
(0015082)
Footkerchief (manager)
2011-02-11 08:38

It would be helpful to post saves that demonstrate this behavior. http://dffd.wimbli.com/ [^]

Overall object count is suspected of causing the biggest FPS hit. Are you able to exclude this factor? I.e., when you have have two fortresses, one with clothes and one without, are the item counts still on par?
(0015086)
Ter13 (reporter)
2011-02-11 09:46
edited on: 2011-02-11 10:05

The idea that item count affects FPS is indicative of overall design issues. Items shouldn't be processed every frame unless there is behavior connected to those items.

The item counts should have been roughly similar, as without clothes, I was able to carry on much longer, and as such produce a much larger quantity of items, all the while my FPS remained stable.

I am very certain that there is a bit of code running unintentionally due to the owned and broken clothing. I believe the dwarf is constantly seeking the nearest replacement for that broken clothing, even though he has a second bit of clothing already. I believe that this is a leading cause to FPS hits, and should be pretty easy to profile and confirm.

(0015104)
Kogut (reporter)
2011-02-14 15:10

How can I disable clothes?
(0015105)
IT 000 (reporter)
2011-02-14 15:40

remove [CLOTHING] and [SUBTERRANEAN_CLOTHING] from the dwarven entity file.
(0015107)
Kogut (reporter)
2011-02-15 02:34

I did this, but on embark dwarfs are still wearing sth. Is it supposed to happen?
(0015122)
hyndis (reporter)
2011-02-16 11:17

I just delete clothing from both the entity files and also in the raws. I leave the armor but clothes are deleted outright. Works fine.
(0016309)
Footkerchief (manager)
2011-03-16 18:19

Reminder sent to: Ter13

We need a save for this report. Please upload one to http://dffd.wimbli.com/ [^]
(0016318)
AbuDhabi (reporter)
2011-03-17 01:32
edited on: 2011-03-17 13:36

Do you think that modifying clothes to have the [HARD] token solve this problem? This would prevent them from rotting at all, like armour does.

EDIT: It seems that just modding out [SOFT] isn't sufficient to battle wear and tear. Though much slower, the clothes do get used up.

(0016816)
Orkel2 (reporter)
2011-03-30 14:21
edited on: 2011-03-30 14:21

This is happening to me too.

Good clothes, ~ 100 FPS on 120 dwarves

Clothes starting to rot, FPS has steadily dropped to 25 in the past year or so.

(0016818)
Footkerchief (manager)
2011-03-30 14:39

Reminder sent to: Orkel2

Please upload a save demonstrating lag from rotting clothes. Bonus points if you can also provide a save from before the clothes start rotting. http://dffd.wimbli.com/ [^]
(0016827)
profit (reporter)
2011-03-31 00:16

I too would really like to see this... I would like to test and see if removing their clothing via putting them into the military with blank uniforms that replace clothing would fix it as well.
(0016830)
profit (reporter)
2011-03-31 00:39

I just tested this on one of my forts, using the military feature I was able to get everyone out of their clothes... (In life I usually just have to ask and they drop) Many of these clothes were completely deteriorated, but I did not experience a FPS gain after I returned the dorfs to civ state even though they did not put on new clothing. ( I forbade it) I am unsure where the FPS boost is coming from but I have a sneaking suspicion it has something to do with the rot code or contaminants not the dorfs trying to replace them.
(0016831)
InsanityPrelude (reporter)
2011-03-31 00:49

Does anything happen if you destroy the rotted clothes? (e.g. throw into magma)
(0016832)
AbuDhabi (reporter)
2011-03-31 01:03

Destroying them is difficult, because they are owned, and therefore can't be dumped.
(0016850)
Kogut (reporter)
2011-03-31 08:01

child of 0002481?
(0016858)
InsanityPrelude (reporter)
2011-03-31 09:21

Oh, right. I forgot about that.
(0017232)
Dekon (reporter)
2011-04-09 08:40

Seems common to me as well. My FPS plummets after any invasion attempt until I can manage to clean up all their crap and put it through an atom smasher. Which means my dwarves end up spending a ludicrous amount of each month just playing clean up from the last invasion.
(0017259)
alcohol_dependent (reporter)
2011-04-09 17:39
edited on: 2011-04-10 10:31

I am going to try removing clothes as well and will post my results.. I have a 3 ghz processor, typically get a big slowdown around 80 dwarves. Never go much beyond there but if I leave temp on with a pop cap at 80 my FPS will slow to 50 with a moderately inefficient fort (35-40 with a lot of designations, or if I decide to style a very inefficient fort).

Will update with results of my next fort. For now I guess, I'll just forge a pair of boots for every-dwarf to sidestep the potential infection problem.


*Update*

Doesn't seem to have made any significant FPS improvements. Much less clutter though. Will tinker more.

(0017337)
kwieland (reporter)
2011-04-13 09:20

I have an established fort (15+ years) 60 dwarfs, half kids. 3x3 embark area. No original clothes left, they've all rotted away. However, all my dwarfs are equiped with cloth cloaks (6), trousers, and socks. These items routinely wear. My normal is 150 FPS. All my dwarfs have rooms with chests and cabinets. Do yours?

My FPS goes down to 40-50 after a siege when I've got a lot of crap to pick up from ground level. I think the FPS hit is due to lots of pathing calculations. Somewhere I read that the path calculation is redone everytime a dwarf runs into another dwarf. Also, during "normal" times, lots of the dwarfs are constructing things, which doesn't call pathing.

My take: my FPS hit is due to pathing not wear.
(0017347)
Footkerchief (manager)
2011-04-13 14:25

Reminder sent to: Orkel2, Ter13

This report will be resolved as "unable to reproduce" in the near future unless saves are uploaded that demonstrate that rotting clothes have a significant impact on FPS, independent of other factors.
(0017434)
devek (reporter)
2011-04-18 17:22

@Ter13

>The idea that item count affects FPS is indicative of overall design issues.

>I am very certain that there is a bit of code running unintentionally due to the
>owned and broken clothing. I believe the dwarf is constantly seeking the nearest
>replacement for that broken clothing, even though he has a second bit of
>clothing already. I believe that this is a leading cause to FPS hits, and should
>be pretty easy to profile and confirm.

How can it be profiled and confirmed if no one ever bothers to upload relevant saves? I think that is more of a design issue with the user base :P

You don't even need Toady to do it, I can use memory hacking to unown all items and then delete them manually to see... if I had a save to work with that is :)
(0017441)
ag (reporter)
2011-04-19 04:09

I tried unowning, dumping into a smasher and pulling the lever on all worn clothes with my new cleanowned tool in DFHack, and it gave at most 10-15% improvement on my save (quite near to the statistical noise level).

Dumping all the cheap junk my stonecrafter created over the years onto a caravan seemed to have a similar effect. The latter was actually inspired by some specific results oprofile gave me, namely by relatively large time spent in binary search to find the container item of an object.
(0017452)
Dekon (reporter)
2011-04-20 20:51

Maybe in a few weeks once classes are all done I'll try and do a fort and collect a crap ton of goblin junk and have it set up for a save, so at least leave this open for a bit longer.

If I've heard correctly (and I may not be correct about this), items have to constantly check themselves to determine 'Am I muddy? Am I on fire? Am I needing to be moved to a stockpile/dump? Am I...' etc etc. Because they're constantly being checked for this, it creates an intense drop in FPS when items start to build dramatically simply because so many things are undergoing this process of self check so frequently.
(0017591)
ag (reporter)
2011-05-01 02:41

After some more profiling and staring at assembly, I have an easy optimization suggestion: reimplement flagarrayst::has_flag as a self-contained inline function without any external calls. It is used a lot for quering flags in raw structures, and currently every usage results in a cross-dll call without any opportunity for the compiler to use the fact that the flag ID is known at compile time. In the linux version get_slot_and_addbit_uchar accounts for 50% of all time spent in libgraphics.so.
(0018727)
gmafk (reporter)
2011-09-15 23:25

I think I have proof that clothing is a source of a significant drop in FPS. My fort (31.25, linux) started at 100 FPS and every migrant wave reduced it until it was running at 30-40 FPS with 120 dwarves.

Save: http://dffd.wimbli.com/file.php?id=4943 [^]

All 120 dwarves are gathered near the garbage dump (F5) and are idle. All armor, legwear, headwear, handwear and footwear is designated for dumping (but dwarves own it so they won't dump it).

Step 1: Do nothing for 10-20s and let the FPS counter stabilize (39 FPS)
Step 2: Run 'cleanowned all' in DFHack
Step 3: Watch your FPS rise as dwarves discard their clothing into lava (54 FPS). This step can take around 2 minutes - you can use the stocks menu to check how many items have been destroyed so far).

Improvement from 39 FPS (25.64 ms) to 54 FPS (18.51 ms) is pretty significant: 7.12 ms spent per frame just to process clothes.
(0018742)
Xen0n (reporter)
2011-09-19 14:55

Perhaps, add a control in case it was just getting rid of so many items that gave the improvement, not necessarily clothes. Take the same save and dump an equal number of crafts (or rocks or something) and compare the FPS improvement to when you dumped all the clothes.
(0018745)
gmafk (reporter)
2011-09-20 18:45

I used the save from my previous post: http://dffd.wimbli.com/file.php?id=4943 [^]

Items other than clothes can be easily tested using 'autodump destroy'. Unfortunately, it doesn't seem to work on equipped items, so I just copied results from my previous test for comparison.

Step 1: Mark items for dumping.
Step 2: Run 'autodump destroy' in DFHack.

The game starts at 39 FPS (25.64 ms) before each test. Results:

Destroying 1,7 k clothes: 54 FPS (18.51 ms), -7.12 ms
Destroying 1,7 k stone (limestone + gabbro): 40 FPS (25.00 ms), -0.64 ms
Destroying 4,5 k stone (all stone on the map): 42 FPS (23.80 ms), -1.83 ms

It seems that processing clothes takes a lot more time than processing rocks.
(0018746)
kwieland (reporter)
2011-09-20 20:07
edited on: 2011-09-20 20:10

gmafk, good work. I think the clothes are PART of the issue, but I think there are other things as well (like contaminants, pathing). I was going to suggest abandoning and reclaiming, but there are other bugs there, so it would be hard to isolate the real cause.

The original bug report states:"Dwarves are constantly checking to see if they need new clothes once they own a broken piece of clothing, and this is causing excess processing to be done" Do you think this is the case? Wouldn't that make your FPS drop after you "liberate" the clothes? Have you run the liberated game for a year to see if the FPS stays up?

(0021488)
thvaz (reporter)
2012-03-15 12:13
edited on: 2012-03-18 00:24

This one shouldn't be fixed by today's devlog?

(0021554)
Kogut (reporter)
2012-03-18 07:56

It is hidden (workaround is to simply produce clothes) but probably if "dwarves wear clothing (we turned it off and tried it), the FPS steadily decreases as more and more dwarves wear out their clothing".
(0021588)
Toady One (administrator)
2012-03-19 04:01

As far as I can tell from having a few hundred dwarves around and rotting their clothing and placing clothing piles and so on, this overall situation is a lot better. That's not to say there aren't going to be problems, especially the ongoing general issues from having lots of objects around, but I'm going to mark this one off.

- Issue History
Date Modified Username Field Change
2011-02-10 15:53 Ter13 New Issue
2011-02-11 08:38 Footkerchief Note Added: 0015082
2011-02-11 08:38 Footkerchief Tag Attached: AWAITING UPDATE
2011-02-11 09:46 Ter13 Note Added: 0015086
2011-02-11 10:05 Ter13 Note Edited: 0015086 View Revisions
2011-02-11 15:29 Hieronymous Alloy Issue Monitored: Hieronymous Alloy
2011-02-14 15:10 Kogut Note Added: 0015104
2011-02-14 15:10 Kogut Issue Monitored: Kogut
2011-02-14 15:40 IT 000 Note Added: 0015105
2011-02-15 02:34 Kogut Note Added: 0015107
2011-02-16 09:35 Naros Issue Monitored: Naros
2011-02-16 11:17 hyndis Note Added: 0015122
2011-03-16 12:06 Footkerchief Relationship added related to 0004207
2011-03-16 15:32 Malibu Stacey Issue Monitored: Malibu Stacey
2011-03-16 18:19 Footkerchief Note Added: 0016309
2011-03-17 01:32 AbuDhabi Note Added: 0016318
2011-03-17 01:32 AbuDhabi Issue Monitored: AbuDhabi
2011-03-17 13:36 AbuDhabi Note Edited: 0016318 View Revisions
2011-03-30 14:21 Orkel2 Note Added: 0016816
2011-03-30 14:21 Orkel2 Note Edited: 0016816 View Revisions
2011-03-30 14:39 Footkerchief Issue Monitored: Orkel2
2011-03-30 14:39 Footkerchief Note Added: 0016818
2011-03-31 00:16 profit Note Added: 0016827
2011-03-31 00:39 profit Note Added: 0016830
2011-03-31 00:49 InsanityPrelude Note Added: 0016831
2011-03-31 00:49 InsanityPrelude Issue Monitored: InsanityPrelude
2011-03-31 01:03 AbuDhabi Note Added: 0016832
2011-03-31 08:01 Kogut Note Added: 0016850
2011-03-31 09:21 InsanityPrelude Note Added: 0016858
2011-03-31 09:32 Footkerchief Relationship added related to 0002481
2011-04-09 08:40 Dekon Note Added: 0017232
2011-04-09 17:39 alcohol_dependent Note Added: 0017259
2011-04-09 17:41 alcohol_dependent Note Edited: 0017259 View Revisions
2011-04-09 18:22 alcohol_dependent Note Edited: 0017259 View Revisions
2011-04-09 19:26 alcohol_dependent Note Edited: 0017259 View Revisions
2011-04-10 10:31 alcohol_dependent Note Edited: 0017259 View Revisions
2011-04-13 09:20 kwieland Note Added: 0017337
2011-04-13 14:25 Footkerchief Note Added: 0017347
2011-04-18 17:22 devek Note Added: 0017434
2011-04-19 04:09 ag Note Added: 0017441
2011-04-20 20:51 Dekon Note Added: 0017452
2011-05-01 02:41 ag Note Added: 0017591
2011-07-09 03:55 Beeskee Issue Monitored: Beeskee
2011-09-14 14:15 eastc Issue Monitored: eastc
2011-09-15 23:25 gmafk Note Added: 0018727
2011-09-16 14:06 Morrolan Issue Monitored: Morrolan
2011-09-19 14:55 Xen0n Note Added: 0018742
2011-09-20 18:45 gmafk Note Added: 0018745
2011-09-20 20:07 kwieland Note Added: 0018746
2011-09-20 20:09 kwieland Note Edited: 0018746 View Revisions
2011-09-20 20:10 kwieland Note Edited: 0018746 View Revisions
2012-02-26 11:46 Buglist Issue Monitored: Buglist
2012-02-29 01:25 kento Issue Monitored: kento
2012-03-15 12:13 thvaz Note Added: 0021488
2012-03-18 00:24 thvaz Note Edited: 0021488 View Revisions
2012-03-18 07:23 Dwarfu Tag Detached: AWAITING UPDATE
2012-03-18 07:24 Dwarfu Issue Monitored: Toady One
2012-03-18 07:56 Kogut Note Added: 0021554
2012-03-19 04:01 Toady One Note Added: 0021588
2012-03-19 04:01 Toady One Status new => resolved
2012-03-19 04:01 Toady One Fixed in Version => Next Version
2012-03-19 04:01 Toady One Resolution open => fixed
2012-03-19 04:01 Toady One Assigned To => Toady One
2012-03-21 15:07 Kogut Issue End Monitor: Kogut
2012-03-23 05:00 Buglist Issue End Monitor: Buglist
2012-03-29 05:39 Naros Issue End Monitor: Naros
2012-03-30 10:21 AbuDhabi Issue End Monitor: AbuDhabi
2012-05-28 17:53 Morrolan Issue End Monitor: Morrolan
2012-07-05 06:46 Malibu Stacey Issue End Monitor: Malibu Stacey
2013-06-07 18:02 InsanityPrelude Issue End Monitor: InsanityPrelude


Copyright © 2000 - 2010 MantisBT Group
Powered by Mantis Bugtracker