0003963: World Generation too slow - Dwarf Fortress Bug Tracker

(0015128)
Quietust (reporter)
2011-02-16 13:23

More importantly, worldgen seems to get exponentially slower the longer it runs.

(0015218)
cephalo (reporter)
2011-02-17 17:33

Sounds to me like the economic stuff churning away. There's probably alot more going on in world gen these days.

(0018401)
asmodai (reporter)
2011-07-31 23:03
edited on: 2011-07-31 23:05

I can definitely confirm this. You see every year that's being generated drop the speed at which it's generated visually. If you have FPS displayed you will notice it drops to 1 FPS after 160 years. It definitely seems to be quadratic or exponential growth.

For me it makes the game pretty much unplayable.

Edit: and that's with 0.31.25

(0018405)
darkteal (reporter)
2011-08-01 01:51

If you don't like it, use a previous version to generate the world. Back in the day, it used to take nearly a week. (Creationist humor, anyone?)

(0018407)
asmodai (reporter)
2011-08-01 07:14

Well, I am positive there's room for improvement in the generation algorithms (and that's the programmer in me speaking). I am used to generate a few worlds at first before settling on one I find appealing for one reason or another. With the latest version taking roughly 15 minutes, it makes that scenario a bit problematic. Not to mention the amount of resident memory the process now uses when it is creating this (500+ MB).

Here's a comparison I did (times are average over 3 gens on a Intel Core 2 Duo @ 2.8 GHz / 4 GB):

0.31.01 - Good (medium region takes about 1:45)
0.31.06 - Good (medium region takes about 1:45)
0.31.08 - Good (medium region takes about 2:00)
0.31.09 - Ok (medium region takes about 2:38)
0.31.10 - Ok (medium region takes about 2:55)
0.31.11 - Good (medium region takes about 2:00)
0.31.12 - Good (medium region takes about 1:55)
0.31.13 - Bad (medium region takes about 8:52)
0.31.24 - Bad (medium region takes about 15:17)

(0019538)
Stromko (reporter)
2012-02-14 11:59

I've been waiting more than an hour for a medium region to be generated on a 2.8 ghz dual-core processor. It's gotten to the year 316 already (I thought I set it to stop at 300), and it will not stop or pause no matter what input I try.

(0019541)
snow (reporter)
2012-02-14 12:11
edited on: 2012-02-14 12:12

i suggest 200 years for convenience. Last i ran to 270 years it already filled 770mb ram. I'm pretty sure it was going to crash around 1gb anyway...

I doubt toady uses the presets for his testing/playing purposes?

(0019563)
king doom (reporter)
2012-02-14 16:04
edited on: 2012-02-14 16:13

I've been creating a new world for about eight hours now - year 297 and over five million events recorded. The longer the world generation goes on the more people there are to generate events about and they have children and so on and it seems like the amount of stuff being recorded is increasing geometrically.

Edit: world gen finished at year 300 like it was supposed to, it recorded 5005742 events.

(0019566)
fonikz (reporter)
2012-02-14 16:54

It's exponentially slower than it was in 31.25. I could make a world with all the defaults in 3 minutes 46 seconds. Now, as others are reporting, it's taking several hours, I end up putting something heavy on my ENTER key to finally get it to stop and let me play.

(0019681)
Lac (reporter)
2012-02-15 14:25

For completeness I'll cf a Toady One quote: "Yeah, world gen is slower. I can still squeeze something out of it. It's mostly the goods being shuffled around and processed as far as I can tell."

(0019883)
VDOgamez (reporter)
2012-02-16 17:47

On my Windows 7 computer, it keeps triggering the "not responding" error messages, which sometimes will screw it up. And when I realize that it's not going to make it to the end year I have set, I can't make it abort because it's freezing up and not taking keypresses. I know the game is single threaded, but I wish that there was some extra thread at least for letting me quit when the generator freezes. Despite having an i7 with 16 GB of RAM, it takes at least an hour to even get to the 250 year range, and crashes soon after that. It can go a bit longer if I shrink the world size down and set stricter population limits, but the world isn't much fun if there's nobody there. I don't know how I'll ever make it to the 1050 years the default settings seem to expect.

(0020101)
MaXMC (reporter)
2012-02-19 01:09

So it's finally time for DF to go Multi-threaded...

(0020103)
DarthCloakedDwarf (reporter)
2012-02-19 01:37

I've noticed that it goes slower the longer history it has.

(0020382)
xaldin (reporter)
2012-02-21 07:59
edited on: 2012-02-21 08:00

Since mine was flagged duplicate I'll post the information here:

Essentially werecurses are the key problem. If you set them to 0 it can do a year per second on a modern system. Werecurses bring even high end systems to a halt after a couple hundred years of generation. See 0005344

(0020395)
NW_Kohaku (reporter)
2012-02-21 10:07

Hmm... this "worldgen too slow" is from 0.31.19, though, not the new version, so werecritters wouldn't have even been in during this report.

I made a report, myself, but it was specifically about the millions of werecreature attcks taking place in worldgen - http://www.bay12games.com/dwarves/mantisbt/view.php?id=5269 [^]

(0020403)
thvaz (reporter)
2012-02-21 12:24

Since 31.19 generation of medium worlds or larger was really slow. 34.01 just made it worse.

(0020433)
xaldin (reporter)
2012-02-21 22:52

Worldgen has been 'slow'er each version. This one that i reported and had closed as a duplicate of this essentially is worldgen stopping/crashing for all intents caused by werecreature growth.

Ticket handling should have flagged yours or mine as a dup and attached them together into one since 5269 is really closer to what 5344 was about. I just didn't want the testing lost as it gives Toady a place to start looking, assuming this is ever addressed.

(0020521)
NW_Kohaku (reporter)
2012-02-22 22:59

I am trying to start a suggestion thread to get some people brainstorming some solutions to slow worldgens. See here: http://www.bay12forums.com/smf/index.php?topic=102293.0 [^]

(0021205)
thvaz (reporter)
2012-03-07 02:10
edited on: 2012-03-07 02:27

Someone still think the world generation is too slow? As of 34.05, I think it's fine now.

(0021215)
Footkerchief (manager)
2012-03-07 08:04

Thanks for the update. I'll mark this one fixed -- reusing an old report for a new problem was probably not the best idea anyway. If new problems arise, we can handle it in a new report.

Issue History
Date Modified	Username	Field	Change
2011-02-16 13:10	thvaz	New Issue
2011-02-16 13:23	Quietust	Note Added: 0015128
2011-02-17 09:35	Khym Chanur	Issue Monitored: Khym Chanur
2011-02-17 17:33	cephalo	Note Added: 0015218
2011-03-06 05:20	thvaz	Tag Attached: Intentional/Expected?
2011-03-06 05:20	thvaz	Tag Attached: Needs Confirmation
2011-07-31 23:03	asmodai	Note Added: 0018401
2011-07-31 23:05	asmodai	Note Edited: 0018401	View Revisions
2011-08-01 00:43	asmodai	Issue Monitored: asmodai
2011-08-01 01:51	darkteal	Note Added: 0018405
2011-08-01 07:14	asmodai	Note Added: 0018407
2012-02-14 11:59	Stromko	Note Added: 0019538
2012-02-14 12:09	Footkerchief	Tag Detached: Intentional/Expected?
2012-02-14 12:09	Footkerchief	Tag Detached: Needs Confirmation
2012-02-14 12:11	snow	Note Added: 0019541
2012-02-14 12:12	snow	Note Edited: 0019541	View Revisions
2012-02-14 16:04	king doom	Note Added: 0019563
2012-02-14 16:13	king doom	Note Edited: 0019563	View Revisions
2012-02-14 16:54	fonikz	Note Added: 0019566
2012-02-15 07:09	Footkerchief	Relationship added	has duplicate 0005096
2012-02-15 14:25	Lac	Note Added: 0019681
2012-02-15 14:47	Footkerchief	Relationship added	has duplicate 0005058
2012-02-16 17:47	VDOgamez	Note Added: 0019883
2012-02-19 01:09	MaXMC	Note Added: 0020101
2012-02-19 01:37	DarthCloakedDwarf	Note Added: 0020103
2012-02-20 16:17	Footkerchief	Relationship added	has duplicate 0005344
2012-02-21 07:59	xaldin	Note Added: 0020382
2012-02-21 08:00	Footkerchief	Note Edited: 0020382	View Revisions
2012-02-21 08:01	Footkerchief	Issue Monitored: Toady One
2012-02-21 10:07	NW_Kohaku	Note Added: 0020395
2012-02-21 11:37	Footkerchief	Relationship added	related to 0005269
2012-02-21 12:24	thvaz	Note Added: 0020403
2012-02-21 22:52	xaldin	Note Added: 0020433
2012-02-22 16:05	xaldin	Issue Monitored: xaldin
2012-02-22 22:59	NW_Kohaku	Note Added: 0020521
2012-03-07 02:10	thvaz	Note Added: 0021205
2012-03-07 02:27	thvaz	Note Edited: 0021205	View Revisions
2012-03-07 08:04	Footkerchief	Note Added: 0021215
2012-03-07 08:04	Footkerchief	Status	new => resolved
2012-03-07 08:04	Footkerchief	Fixed in Version	=> 0.34.03
2012-03-07 08:04	Footkerchief	Resolution	open => fixed
2012-03-07 08:04	Footkerchief	Assigned To	=> Toady One

Notes
(0015128) Quietust (reporter) 2011-02-16 13:23	More importantly, worldgen seems to get exponentially slower the longer it runs.

(0015218) cephalo (reporter) 2011-02-17 17:33	Sounds to me like the economic stuff churning away. There's probably alot more going on in world gen these days.

(0018401) asmodai (reporter) 2011-07-31 23:03 edited on: 2011-07-31 23:05	I can definitely confirm this. You see every year that's being generated drop the speed at which it's generated visually. If you have FPS displayed you will notice it drops to 1 FPS after 160 years. It definitely seems to be quadratic or exponential growth. For me it makes the game pretty much unplayable. Edit: and that's with 0.31.25

(0018405) darkteal (reporter) 2011-08-01 01:51	If you don't like it, use a previous version to generate the world. Back in the day, it used to take nearly a week. (Creationist humor, anyone?)

(0018407) asmodai (reporter) 2011-08-01 07:14	Well, I am positive there's room for improvement in the generation algorithms (and that's the programmer in me speaking). I am used to generate a few worlds at first before settling on one I find appealing for one reason or another. With the latest version taking roughly 15 minutes, it makes that scenario a bit problematic. Not to mention the amount of resident memory the process now uses when it is creating this (500+ MB). Here's a comparison I did (times are average over 3 gens on a Intel Core 2 Duo @ 2.8 GHz / 4 GB): 0.31.01 - Good (medium region takes about 1:45) 0.31.06 - Good (medium region takes about 1:45) 0.31.08 - Good (medium region takes about 2:00) 0.31.09 - Ok (medium region takes about 2:38) 0.31.10 - Ok (medium region takes about 2:55) 0.31.11 - Good (medium region takes about 2:00) 0.31.12 - Good (medium region takes about 1:55) 0.31.13 - Bad (medium region takes about 8:52) 0.31.24 - Bad (medium region takes about 15:17)

(0019538) Stromko (reporter) 2012-02-14 11:59	I've been waiting more than an hour for a medium region to be generated on a 2.8 ghz dual-core processor. It's gotten to the year 316 already (I thought I set it to stop at 300), and it will not stop or pause no matter what input I try.

(0019541) snow (reporter) 2012-02-14 12:11 edited on: 2012-02-14 12:12	i suggest 200 years for convenience. Last i ran to 270 years it already filled 770mb ram. I'm pretty sure it was going to crash around 1gb anyway... I doubt toady uses the presets for his testing/playing purposes?

(0019563) king doom (reporter) 2012-02-14 16:04 edited on: 2012-02-14 16:13	I've been creating a new world for about eight hours now - year 297 and over five million events recorded. The longer the world generation goes on the more people there are to generate events about and they have children and so on and it seems like the amount of stuff being recorded is increasing geometrically. Edit: world gen finished at year 300 like it was supposed to, it recorded 5005742 events.

(0019566) fonikz (reporter) 2012-02-14 16:54	It's exponentially slower than it was in 31.25. I could make a world with all the defaults in 3 minutes 46 seconds. Now, as others are reporting, it's taking several hours, I end up putting something heavy on my ENTER key to finally get it to stop and let me play.

(0019681) Lac (reporter) 2012-02-15 14:25	For completeness I'll cf a Toady One quote: "Yeah, world gen is slower. I can still squeeze something out of it. It's mostly the goods being shuffled around and processed as far as I can tell."

(0019883) VDOgamez (reporter) 2012-02-16 17:47	On my Windows 7 computer, it keeps triggering the "not responding" error messages, which sometimes will screw it up. And when I realize that it's not going to make it to the end year I have set, I can't make it abort because it's freezing up and not taking keypresses. I know the game is single threaded, but I wish that there was some extra thread at least for letting me quit when the generator freezes. Despite having an i7 with 16 GB of RAM, it takes at least an hour to even get to the 250 year range, and crashes soon after that. It can go a bit longer if I shrink the world size down and set stricter population limits, but the world isn't much fun if there's nobody there. I don't know how I'll ever make it to the 1050 years the default settings seem to expect.

(0020101) MaXMC (reporter) 2012-02-19 01:09	So it's finally time for DF to go Multi-threaded...

(0020103) DarthCloakedDwarf (reporter) 2012-02-19 01:37	I've noticed that it goes slower the longer history it has.

(0020382) xaldin (reporter) 2012-02-21 07:59 edited on: 2012-02-21 08:00	Since mine was flagged duplicate I'll post the information here: Essentially werecurses are the key problem. If you set them to 0 it can do a year per second on a modern system. Werecurses bring even high end systems to a halt after a couple hundred years of generation. See 0005344

(0020395) NW_Kohaku (reporter) 2012-02-21 10:07	Hmm... this "worldgen too slow" is from 0.31.19, though, not the new version, so werecritters wouldn't have even been in during this report. I made a report, myself, but it was specifically about the millions of werecreature attcks taking place in worldgen - http://www.bay12games.com/dwarves/mantisbt/view.php?id=5269 [^]

(0020403) thvaz (reporter) 2012-02-21 12:24	Since 31.19 generation of medium worlds or larger was really slow. 34.01 just made it worse.

(0020433) xaldin (reporter) 2012-02-21 22:52	Worldgen has been 'slow'er each version. This one that i reported and had closed as a duplicate of this essentially is worldgen stopping/crashing for all intents caused by werecreature growth. Ticket handling should have flagged yours or mine as a dup and attached them together into one since 5269 is really closer to what 5344 was about. I just didn't want the testing lost as it gives Toady a place to start looking, assuming this is ever addressed.

(0020521) NW_Kohaku (reporter) 2012-02-22 22:59	I am trying to start a suggestion thread to get some people brainstorming some solutions to slow worldgens. See here: http://www.bay12forums.com/smf/index.php?topic=102293.0 [^]

(0021205) thvaz (reporter) 2012-03-07 02:10 edited on: 2012-03-07 02:27	Someone still think the world generation is too slow? As of 34.05, I think it's fine now.

(0021215) Footkerchief (manager) 2012-03-07 08:04	Thanks for the update. I'll mark this one fixed -- reusing an old report for a new problem was probably not the best idea anyway. If new problems arise, we can handle it in a new report.