0005703: Underground lake, despite having fish, is always reported as "there is nothing to catch"

(0021727)
Footkerchief (manager)
2012-03-25 19:34

Please upload a save demonstrating this problem to http://dffd.wimbli.com/ [^]

(0021730)
thvaz (reporter)
2012-03-25 23:35

I lost that fortress to a epic tantrum spiral, but I will set up a test fortress and will provide the save.

(0021799)
thvaz (reporter)
2012-03-27 15:38

I made a test game for another bug, but it will serve for this one too, though this time I never saw the blinking cave fish. The isn't any fish at all in the map - nothing at the pools nor at the underground lakes.
http://dffd.wimbli.com/file.php?id=6018 [^]

(0024195)
UristDaVinci (reporter)
2013-11-05 23:33

The code which checks the population list to find fish that could be caught doesn't define the region_x and region_y of the map block of the water being fished in, if the water lies in a global feature (such as a cavern). Region_x and Region_y are defined if the water is in a local feature, such as a river/stream.

The code which determines if a fish is eligible to be caught appears to check the region_x, region_y, local feature index, and global feature index of the fish against the properties of the map block. A cave fish in one of my games requires this:
region_x = 10
region_y = 8
local_idx = -1
global_idx = 2
however, the region_x and region_y are always filled with garbage data when a tile is only designated as a global feature.

(0024196)
smjjames (reporter)
2013-11-06 00:17

@uristdavinci: Could that have a role in the problems with fishing on the surface? Mainly the not always being able to fish in locations rather than the extinction, though it could play a role in the fish populations going extinct.

Though I heard that the masterwork mod (or someone) kind of solved the extinction problem by giving the vermin fish huge pop numbers.

Theres all kinds of population changes in the next version, so we will just have to wait and see on that.

(0024197)
Quietust (reporter)
2013-11-06 05:42
edited on: 2013-11-06 05:43

The only information necessary to determine the populations within a global feature (technically an "underground region") is the underground region ID itself, since the underground region is where the populations are stored and its ID does not depend on location within the world (unlike local features, where the game has a table of lists for all region coordinates).

(0024198)
UristDaVinci (reporter)
2013-11-06 17:49

@Quietust: YES, but the relevant code is something like this:

if(*(eax + 0xc) == region_x && *(eax + 0xe) == region_y) {
    if(*(eax + 0x10) == local_feature && *(eax + 0x14) == global_feature && *(eax + 0x18) == -1 && *(eax + 4) > 0 && *eax == 1) {
        $check here to see if it is a vermin fish
    }
}

The population must have the correct location within the world before the game even checks to see if it matches the underground region ID.

Earlier in the code, the game assumes that the location doesn't matter if the tile being fished is in a global feature (underground region), and leaves the variables undefined. There is a 1/4294836225 = 0.00000002% chance that you could catch a cave fish, depending on the random contents of the memory.

(0024199)
smjjames (reporter)
2013-11-06 21:06

Well okay, but does it apply in the same way for surface fish vermin or does it do it differently?

(0024200)
Quietust (reporter)
2013-11-07 05:37

If that's what the code is doing, then it's almost definitely wrong - at what address is that particular logic?

(0024201)
UristDaVinci (reporter)
2013-11-07 19:45

@Quietust: I am using Microsoft WinDbg

The "check" logic starts at Dwarf_Fortress+0x6697a1

The "if in local region, define region_x and region_y" logic starts at Dwarf_Fortress+0x75cc27

In WinDbg I can just use the command "bp Dwarf_Fortress+0ABCDE" to set a breakpoint.

(0024202)
smjjames (reporter)
2013-11-08 05:36

Even though I'm mostly lost in your technical discussion of the code, I'm still wondering whether the problem as described by UristDaVinci also applies to surface vermin fish. So, does it? Sounds like it could explain some features of the related bugs with surface vermin fish.

Also <1% chance? wow.

(0024204)
Quietust (reporter)
2013-11-10 08:57
edited on: 2013-11-11 14:17

From what I can tell, this is what's happening:
1. It processes the job coordinates to determine the tile being fished from.
2. It looks at that tile to determine local feature ID (default -1) and and region coordinates (default undefined), then global feature ID/subID (default -1). Region coordinates are only set if a local feature is present.
3. If no local feature or global feature is present, it explicitly calculates the region coordinates.
4. It tries to match those values against all local populations (using the logic in UristDaVinci's comment above).

The following binary patch should disable the second check in step 3 (so it only checks for the absence of a local feature) on Win32 DF 0.34.11 SDL and allow this to work correctly:

0x668B3B : 14 -> 00

I have not yet tested this patch - if anybody here could do so, it would be greatly appreciated.

(0024205)
Quietust (reporter)
2013-11-10 09:29
edited on: 2013-11-10 09:44

Corresponding (and also untested) patch for 0.34.11 Linux:

0xB18D71 : 68 FF FF FF -> 00 00 00 00

...and for 0.34.11 OSX:

0xACB6A8 : D9 -> 00

(0024206)
UristDaVinci (reporter)
2013-11-10 16:03

I tested Quietust's binary patch for Win32 DF 0.34.11 SDL (only, not the linux or osx versions) and it works.

I created a new world with [POPULATION_NUMBER:25000:50000] for cave fish, since the default 250:500 can result in 0 or 1 cave fish per region. Ditto for cave lobsters. According to pre-fishing population checks, 21 fish were available. Fish could be seen in the underground cavern water. Dug straight down to the cavern after embarking, and successfully collected a raw cave fish from the lake.

Comment: People may still complain about no fish underground because of the low population counts.

(0024207)
smjjames (reporter)
2013-11-11 06:33
edited on: 2013-11-11 06:34

Are you two ignoring the question(s) I keep aiming towards you?

The fact that Toady One made wildlife populations (and by extension, vermin fish, hopefully) regenerate over time along with using a ridiulously huge pop number should help regenerate the fish, though probably very slowly if the local populations are really low.

Also, how do I do pre-fishing population checks?

(0024210)
Quietust (reporter)
2013-11-11 14:22
edited on: 2013-11-11 14:24

smjjames: this bug should have no effect on surface fishing, since rivers are local features (and thus always initialize the region coordinates) and murky pools are just part of the local biome.

This bug also has nothing to do with depletion of wildlife populations - the problem is that although the populations are present and non-depleted, the job code is failing to locate them due to invalid search criteria.

(0024211)
smjjames (reporter)
2013-11-11 14:32

Okay, was just wondering about it.

Issue History
Date Modified	Username	Field	Change
2012-03-25 08:58	thvaz	New Issue
2012-03-25 19:34	Footkerchief	Note Added: 0021727
2012-03-25 19:34	Footkerchief	Relationship added	related to 0001854
2012-03-25 19:35	Footkerchief	Relationship added	related to 0000278
2012-03-25 19:35	Footkerchief	Relationship deleted	related to 0000278
2012-03-25 19:35	Footkerchief	Relationship added	related to 0000278
2012-03-25 19:35	Footkerchief	Relationship deleted	related to 0000278
2012-03-25 19:35	Footkerchief	Relationship added	related to 0002780
2012-03-25 23:35	thvaz	Note Added: 0021730
2012-03-27 15:38	thvaz	Note Added: 0021799
2013-11-05 23:33	UristDaVinci	Note Added: 0024195
2013-11-06 00:17	smjjames	Note Added: 0024196
2013-11-06 05:42	Quietust	Note Added: 0024197
2013-11-06 05:43	Quietust	Note Edited: 0024197	View Revisions
2013-11-06 17:49	UristDaVinci	Note Added: 0024198
2013-11-06 21:06	smjjames	Note Added: 0024199
2013-11-07 05:37	Quietust	Note Added: 0024200
2013-11-07 19:45	UristDaVinci	Note Added: 0024201
2013-11-08 05:36	smjjames	Note Added: 0024202
2013-11-10 08:57	Quietust	Note Added: 0024204
2013-11-10 08:57	Quietust	Tag Attached: binary patch
2013-11-10 08:59	Quietust	Note Edited: 0024204	View Revisions
2013-11-10 09:29	Quietust	Note Added: 0024205
2013-11-10 09:34	Quietust	Note Edited: 0024204	View Revisions
2013-11-10 09:44	Quietust	Note Edited: 0024205	View Revisions
2013-11-10 16:03	UristDaVinci	Note Added: 0024206
2013-11-10 16:04	UristDaVinci	Tag Attached: Probable Quick Fix
2013-11-11 06:33	smjjames	Note Added: 0024207
2013-11-11 06:34	smjjames	Note Edited: 0024207	View Revisions
2013-11-11 14:17	Quietust	Note Edited: 0024204	View Revisions
2013-11-11 14:22	Quietust	Note Added: 0024210
2013-11-11 14:24	Quietust	Note Edited: 0024210	View Revisions
2013-11-11 14:32	smjjames	Note Added: 0024211
2014-01-15 14:47	Kirig Stonebeard II	Issue Monitored: Kirig Stonebeard II
2014-01-17 10:09	Kirig Stonebeard	Issue Monitored: Kirig Stonebeard
2014-03-25 13:32	Dwarfu	Assigned To	=> Dwarfu
2014-03-25 13:32	Dwarfu	Status	new => acknowledged
2014-07-23 12:31	Toady One	Status	acknowledged => resolved
2014-07-23 12:31	Toady One	Fixed in Version	=> Next Version
2014-07-23 12:31	Toady One	Resolution	open => fixed
2014-07-23 12:31	Toady One	Assigned To	Dwarfu => Toady One

Notes
(0021727) Footkerchief (manager) 2012-03-25 19:34	Please upload a save demonstrating this problem to http://dffd.wimbli.com/ [^]

(0021730) thvaz (reporter) 2012-03-25 23:35	I lost that fortress to a epic tantrum spiral, but I will set up a test fortress and will provide the save.

(0021799) thvaz (reporter) 2012-03-27 15:38	I made a test game for another bug, but it will serve for this one too, though this time I never saw the blinking cave fish. The isn't any fish at all in the map - nothing at the pools nor at the underground lakes. http://dffd.wimbli.com/file.php?id=6018 [^]

(0024195) UristDaVinci (reporter) 2013-11-05 23:33	The code which checks the population list to find fish that could be caught doesn't define the region_x and region_y of the map block of the water being fished in, if the water lies in a global feature (such as a cavern). Region_x and Region_y are defined if the water is in a local feature, such as a river/stream. The code which determines if a fish is eligible to be caught appears to check the region_x, region_y, local feature index, and global feature index of the fish against the properties of the map block. A cave fish in one of my games requires this: region_x = 10 region_y = 8 local_idx = -1 global_idx = 2 however, the region_x and region_y are always filled with garbage data when a tile is only designated as a global feature.

(0024196) smjjames (reporter) 2013-11-06 00:17	@uristdavinci: Could that have a role in the problems with fishing on the surface? Mainly the not always being able to fish in locations rather than the extinction, though it could play a role in the fish populations going extinct. Though I heard that the masterwork mod (or someone) kind of solved the extinction problem by giving the vermin fish huge pop numbers. Theres all kinds of population changes in the next version, so we will just have to wait and see on that.

(0024197) Quietust (reporter) 2013-11-06 05:42 edited on: 2013-11-06 05:43	The only information necessary to determine the populations within a global feature (technically an "underground region") is the underground region ID itself, since the underground region is where the populations are stored and its ID does not depend on location within the world (unlike local features, where the game has a table of lists for all region coordinates).

(0024198) UristDaVinci (reporter) 2013-11-06 17:49	@Quietust: YES, but the relevant code is something like this: if((eax + 0xc) == region_x && (eax + 0xe) == region_y) { if((eax + 0x10) == local_feature && (eax + 0x14) == global_feature && (eax + 0x18) == -1 && (eax + 4) > 0 && *eax == 1) { $check here to see if it is a vermin fish } } The population must have the correct location within the world before the game even checks to see if it matches the underground region ID. Earlier in the code, the game assumes that the location doesn't matter if the tile being fished is in a global feature (underground region), and leaves the variables undefined. There is a 1/4294836225 = 0.00000002% chance that you could catch a cave fish, depending on the random contents of the memory.

(0024199) smjjames (reporter) 2013-11-06 21:06	Well okay, but does it apply in the same way for surface fish vermin or does it do it differently?

(0024200) Quietust (reporter) 2013-11-07 05:37	If that's what the code is doing, then it's almost definitely wrong - at what address is that particular logic?

(0024201) UristDaVinci (reporter) 2013-11-07 19:45	@Quietust: I am using Microsoft WinDbg The "check" logic starts at Dwarf_Fortress+0x6697a1 The "if in local region, define region_x and region_y" logic starts at Dwarf_Fortress+0x75cc27 In WinDbg I can just use the command "bp Dwarf_Fortress+0ABCDE" to set a breakpoint.

(0024202) smjjames (reporter) 2013-11-08 05:36	Even though I'm mostly lost in your technical discussion of the code, I'm still wondering whether the problem as described by UristDaVinci also applies to surface vermin fish. So, does it? Sounds like it could explain some features of the related bugs with surface vermin fish. Also <1% chance? wow.

(0024204) Quietust (reporter) 2013-11-10 08:57 edited on: 2013-11-11 14:17	From what I can tell, this is what's happening: 1. It processes the job coordinates to determine the tile being fished from. 2. It looks at that tile to determine local feature ID (default -1) and and region coordinates (default undefined), then global feature ID/subID (default -1). Region coordinates are only set if a local feature is present. 3. If no local feature or global feature is present, it explicitly calculates the region coordinates. 4. It tries to match those values against all local populations (using the logic in UristDaVinci's comment above). The following binary patch should disable the second check in step 3 (so it only checks for the absence of a local feature) on Win32 DF 0.34.11 SDL and allow this to work correctly: 0x668B3B : 14 -> 00 I have not yet tested this patch - if anybody here could do so, it would be greatly appreciated.

(0024205) Quietust (reporter) 2013-11-10 09:29 edited on: 2013-11-10 09:44	Corresponding (and also untested) patch for 0.34.11 Linux: 0xB18D71 : 68 FF FF FF -> 00 00 00 00 ...and for 0.34.11 OSX: 0xACB6A8 : D9 -> 00

(0024206) UristDaVinci (reporter) 2013-11-10 16:03	I tested Quietust's binary patch for Win32 DF 0.34.11 SDL (only, not the linux or osx versions) and it works. I created a new world with [POPULATION_NUMBER:25000:50000] for cave fish, since the default 250:500 can result in 0 or 1 cave fish per region. Ditto for cave lobsters. According to pre-fishing population checks, 21 fish were available. Fish could be seen in the underground cavern water. Dug straight down to the cavern after embarking, and successfully collected a raw cave fish from the lake. Comment: People may still complain about no fish underground because of the low population counts.

(0024207) smjjames (reporter) 2013-11-11 06:33 edited on: 2013-11-11 06:34	Are you two ignoring the question(s) I keep aiming towards you? The fact that Toady One made wildlife populations (and by extension, vermin fish, hopefully) regenerate over time along with using a ridiulously huge pop number should help regenerate the fish, though probably very slowly if the local populations are really low. Also, how do I do pre-fishing population checks?

(0024210) Quietust (reporter) 2013-11-11 14:22 edited on: 2013-11-11 14:24	smjjames: this bug should have no effect on surface fishing, since rivers are local features (and thus always initialize the region coordinates) and murky pools are just part of the local biome. This bug also has nothing to do with depletion of wildlife populations - the problem is that although the populations are present and non-depleted, the job code is failing to locate them due to invalid search criteria.

(0024211) smjjames (reporter) 2013-11-11 14:32	Okay, was just wondering about it.