Skip to content

Commit ca5f2aa

Browse files
fm3daniel-werknollengewaechsphilippottoMichaelBuessemeyer
authored
In Proofreading, Load Oversegmentation, Perform Merges Eagerly in Frontend (#7654)
* add datastore route to list all agglomerate ids * remove unused file access * types * Apply hdf5 mappings in frontend [WIP] * Fix segment and agglomerate id mixup in proofreading saga * add route to tracingstore * edit mapping after merge and set it in redux store * Fix mapping initialization * WIP: implement split in frontend * Only load subset of mapping for segments visible 5s after page load * small fix * cleanup, time measurements, throttle mapping requests to 500ms -> currently too laggy * Update redux-saga to use throttle with effectChannels * Only refresh mapping if bucket data changed. Fix partial mapping updates after master merge. * Fix how proofreading actions update the mapping in the frontend * disable most ci checks * misc improvements * refine shouldUseDataStore logic * fix type error * fix some type errors * fix more type errors * use NumberLike type for number and bigint at various places * more NumberLike usages * fix lots of typescript errors (related to bigint and to redux) * fix more type errors * fix the last type errors * restore proper NumberLike definition and fix remaining type errors * remove unused imports * fix race condition which could cause mapping to not be properly initialized upon reload * only pass isCentered instead of centeredSegmentId to SegmentListItems * fix react warning when closing context menu for the first time * add debug code for rare scenario where segment id is not an integer * fix TS error * fix that hovered unmapped segment id would not update sometimes * avoid two map look ups * add dryUpdate step before saving proofreading update actions * undo provoking the error * make selective segment visibility in proofreading an option * fix NaN value after mapping unknown segment key * avoid parallel executions of updateHdf5Mapping and also cancel updateHdf5Mapping if wk enters a busy state to avoid using outdated values * always compare known segment ids to newest mapping instead of 'previous' one that isn't updated when doing proofreading operations * don't crash completely when segment mapping is not known yet and user initiates proofreading action * remove artificial delays * improve logging * fix ts error * remove debugging code for NaN mapped id * tweak hovering * rename MIN_CUT_AGGLOMERATE actions * adapt cutFromNeighbors to magic mapping approach * avoid roundtrip for mesh reloading by using newest mapping * tweak hovering (II) * extract code into gatherInfoForOperation * fix context menu for skeletons in 3d viewport * make agglomerate-skeleton-proofreading compatible with magic mappings * don't run rendering code for context menu when its not open * remove unused previousMappingObject * dynamically switch between local and remotely applied mappings when switching to/from proofreading tool etc. (unfortunately, buggy) * auto-reload page if dev-proxy fails * fix missing reload when disabling/re-enabling mapping (now too many reloads are performed) * extract finishMappingInitialization action * extract ensureMappingsAreLoadedAndRequestedMappingExists * rename mappingIsEditable to hasEditableMapping * introduce BucketRetrievalSource to better cancel/restart bucket watchers and reload the layer when necessary * introduce clearMappingAction * make sure that getBucketRetrievalSource doesn't create new instances all the time when multiple volume layers exist * move cuckoo modules into libs/cuckoo * use cuckoo hashing for gpu-based mapping instead of binary search (proper uint64 support needs to be tested) * add missing module * remove logging * make use of protobuf shortcut getHasEditableMapping * Update frontend/javascripts/oxalis/model/sagas/mapping_saga.ts Co-authored-by: MichaelBuessemeyer <39529669+MichaelBuessemeyer@users.noreply.github.com> * Update frontend/javascripts/oxalis/model/sagas/proofread_saga.ts Co-authored-by: MichaelBuessemeyer <39529669+MichaelBuessemeyer@users.noreply.github.com> * mappingIsEditable -> hasEditableMapping to fix compilation error * remove unnecessary braces * incorporate some pr feedback * more pr feedback * add comment to cuckoo table 64 bit * also implement cuckoo table for uint32 keys and values * fix invalid initialization of mapping texture when mapping is disabled * fix that mapping was applied twice * fix uint32 cuckoo implementation (still hardcoded to always use 32 bit) * use 64 bit look up when necessary in shader * remove unused mappingSize * avoid toolbar rerendering * only write necessary changes to cuckoo table instead of rewriting it from scratch every time * ensure diffing of previous and new mapping is fast be providing cached diff operation for which the cache is manually set by the mapping saga * eagerly compute value set when reasonable to avoid clustering of that computation and to improve FPS * add missing worker * implement maintenance of value set for entire data cube to avoid recomputation * Revert maintenance of value set because it didn't help with performance This reverts commit c7bb508. * remove todo because it was tried in c7bb508 and reverted afterwards * remove some time logging * optimize/combine some set operations (2x as fast now) * move and rename diffSets function * further optimization of set operations (in total 4x faster compared to the initial version) * remove unused code * remove superfluous parameter * add renaming todo * delete create_set_from_array webworker as it didn't show an improved FPS rate (therefore, the overhead of passing the data doesn't seem reasonable) * delete debugging code from cuckoo modules * move attemptMappingLookUp glsl code * add some comments * reactivate CI checks * remove unused code * remove obsolete imports * simplify editableMapping check * fix ts problems * fix cyclic dependencies * fix 64 bit mapping rendering * remove UI warnings that the merger mode doesn't support 64 bit, because now it does * fix linting * fix proto related tests * fix more specs * fix pullqueue spec * refactor setNumberLike in cuckoo tables and update some todo comments * remove unused route, change tracingstore mapping route to proto ListOfLong * toSet * remove unused getAgglomerateIdForSegmentId * fix rendering bug outside of viewport on some GPUs * adapt agglomeratesForSegments route to new protobuf interface * link 64-bit issue (#6921) in todo comments * introduce mappingIsPartial uniform * mention #7895 in todo comment * remove commented code * avoid expensive console.log for large mappings * disable most ci checks * fix incorrect hash table size and use hashCombine twice to fix poor capacity utilization due to suboptimal hash distribution * fix cuckootable rehash (did redundant work) and adapt max iterations parameter * if many inserts are done for the mapping, flush the table at the end * fix weak hashing for power-of-two table sizes * refactor diminished hash capacity tweak; clean up and extend tests so that maxing out capacity is tested, too * clean up reloadData related code in mapping_saga * fix serializing bigint to protobuf long * fix toggling of json mappings * group consecutive actions in action logger middleware; add debug logging for dispatched actions * fix that forceful disabled tool wasn't re-enabled when possible (e.g., after toggling segmentation opacity) * mention 64 bit support issue in comment * fix broken mapping of ids by sorting the input ids for the server * add comment about sorting * don't map ids dynamically in segment list view (instead segment items are created with the mapped id if a mapping exists); see #7919 as a follow-up * test reaching critical capacity and remove todo comment * remove some todo comments regarding mapId code that might return unmapped ids if the mapping is partial (impact should be low, add comments for it) * remove more todos and fix toggling of hdf mappings when no volume tracing exists * cast to number when sorting bigint * use bigint in proofreading_saga where sensible and cast to number as late as possible (e.g., in action creators, in REST API etc) * disable more ci stuff * try to handle most ids as Number and cast to Number only when dealing with mapping object and buckets * show zoom warning for agglomerate files only when the mapping is applied remotely * fix that supervoxel highlighting of mesh stays active after leaving proofreading tool (fixes #7867) * fix that after changing the color of a mesh via the segments tab the mesh is always highlighted after initial hover (fixes #7845) * remove unused imports * fix proper type of values returned from getAgglomeratesForSegmentsFrom* * fix incorrect bigint check and refactor to avoid the same mistake in the future * fix unnecessary type adaption that failed on null * fix another sorting bug * integrate pr feedback * rename cuckoo table to CuckooTableVec3 * unify 64 bit todo comments * forceful -> forcefully * refactor eager value set computation * use 0s to initialize mapping uniforms * refactor/fix mapId logic for unmapped ids * fix selective visibility for alpha != 0.2 * only emit soft errors when a data value could not be mapped * fix mapping message hiding too early/never; fix disabled message in mapping UI * misc console stuff * also skip texture updates for cuckoo table when lots of unsets need to be done * show short user notification when segmentation layer is reloaded * highlight whole segment mesh on hover even when geometry is not merged (i.e., super-voxels are highlightable) if not in proofreading tool * use current mag when reading segment ids in proofreading (unless agglomerate skeletons are used) * remove last todo comments * update changelog * remove console.log * re-enable ci checks * fix linting --------- Co-authored-by: Daniel Werner <daniel.werner@scalableminds.com> Co-authored-by: Charlie Meister <charlie.meister@student.hpi.de> Co-authored-by: Philipp Otto <philipp.4096@gmail.com> Co-authored-by: Philipp Otto <philippotto@users.noreply.github.com> Co-authored-by: MichaelBuessemeyer <39529669+MichaelBuessemeyer@users.noreply.github.com>
1 parent 44007eb commit ca5f2aa

File tree

86 files changed

+2647
-1033
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

86 files changed

+2647
-1033
lines changed

CHANGELOG.unreleased.md

+2
Original file line numberDiff line numberDiff line change
@@ -20,10 +20,12 @@ For upgrade instructions, please check the [migration guide](MIGRATIONS.released
2020
- Upgraded backend dependencies for improved performance and stability. [#7922](https://github.com/scalableminds/webknossos/pull/7922)
2121
- It is now saved whether segment groups are collapsed or expanded, so this information doesn't get lost e.g. upon page reload. [#7928](https://github.com/scalableminds/webknossos/pull/7928/)
2222
- The context menu entry "Focus in Segment List" expands all necessary segment groups in the segments tab to show the highlighted segment. [#7950](https://github.com/scalableminds/webknossos/pull/7950)
23+
- In the proofreading mode, you can enable/disable that only the active segment and the hovered segment are rendered. [#7654](https://github.com/scalableminds/webknossos/pull/7654)
2324

2425
### Changed
2526
- The warning about a mismatch between the scale of a pre-computed mesh and the dataset scale's factor now also considers all supported mags of the active segmentation layer. This reduces the false posive rate regarding this warning. [#7921](https://github.com/scalableminds/webknossos/pull/7921/)
2627
- It is no longer allowed to edit annotations of other organizations, even if they are set to public and to others-may-edit. [#7923](https://github.com/scalableminds/webknossos/pull/7923)
28+
- When proofreading segmentations, the user can now interact with super-voxels directly in the data viewports. Additionally, proofreading is significantly faster because the segmentation data doesn't have to be re-downloaded after each merge/split operation. [#7654](https://github.com/scalableminds/webknossos/pull/7654)
2729

2830
### Fixed
2931
- Fixed a bug that allowed the default newly created bounding box to appear outside the dataset. In case the whole bounding box would be outside it is created regardless. [#7892](https://github.com/scalableminds/webknossos/pull/7892)

app/models/annotation/nml/NmlWriter.scala

+1-1
Original file line numberDiff line numberDiff line change
@@ -263,7 +263,7 @@ class NmlWriter @Inject()(implicit ec: ExecutionContext) extends FoxImplicits {
263263
case Right(volumeTracing) =>
264264
volumeTracing.fallbackLayer.foreach(writer.writeAttribute("fallbackLayer", _))
265265
volumeTracing.largestSegmentId.foreach(id => writer.writeAttribute("largestSegmentId", id.toString))
266-
if (!volumeTracing.mappingIsEditable.getOrElse(false)) {
266+
if (!volumeTracing.hasEditableMapping.getOrElse(false)) {
267267
volumeTracing.mappingName.foreach { mappingName =>
268268
writer.writeAttribute("mappingName", mappingName)
269269
}

frontend/javascripts/admin/admin_rest_api.ts

+87-26
Original file line numberDiff line numberDiff line change
@@ -77,13 +77,19 @@ import type {
7777
MappingType,
7878
VolumeTracing,
7979
UserConfiguration,
80+
Mapping,
81+
NumberLike,
8082
} from "oxalis/store";
8183
import type { NewTask, TaskCreationResponseContainer } from "admin/task/task_create_bulk_view";
8284
import type { QueryObject } from "admin/task/task_search_form";
8385
import { V3 } from "libs/mjs";
8486
import type { Versions } from "oxalis/view/version_view";
8587
import { enforceValidatedDatasetViewConfiguration } from "types/schemas/dataset_view_configuration_defaults";
86-
import { parseProtoTracing } from "oxalis/model/helpers/proto_helpers";
88+
import {
89+
parseProtoListOfLong,
90+
parseProtoTracing,
91+
serializeProtoListOfLong,
92+
} from "oxalis/model/helpers/proto_helpers";
8793
import type { RequestOptions } from "libs/request";
8894
import Request from "libs/request";
8995
import type { Message } from "libs/toast";
@@ -886,7 +892,7 @@ export async function getTracingForAnnotationType(
886892
): Promise<ServerTracing> {
887893
const { tracingId, typ } = annotationLayerDescriptor;
888894
const version = extractVersion(versions, tracingId, typ);
889-
const tracingType = typ.toLowerCase();
895+
const tracingType = typ.toLowerCase() as "skeleton" | "volume";
890896
const possibleVersionString = version != null ? `&version=${version}` : "";
891897
const tracingArrayBuffer = await doWithToken((token) =>
892898
Request.receiveArraybuffer(
@@ -1599,23 +1605,6 @@ export function getEditableMappingInfo(
15991605
);
16001606
}
16011607

1602-
export function getAgglomerateIdForSegmentId(
1603-
tracingStoreUrl: string,
1604-
tracingId: string,
1605-
segmentId: number,
1606-
): Promise<number> {
1607-
return doWithToken(async (token) => {
1608-
const urlParams = new URLSearchParams({
1609-
token,
1610-
segmentId: `${segmentId}`,
1611-
});
1612-
const { agglomerateId } = await Request.receiveJSON(
1613-
`${tracingStoreUrl}/tracings/mapping/${tracingId}/agglomerateIdForSegmentId?${urlParams.toString()}`,
1614-
);
1615-
return agglomerateId;
1616-
});
1617-
}
1618-
16191608
export function getPositionForSegmentInAgglomerate(
16201609
datastoreUrl: string,
16211610
datasetId: APIDatasetId,
@@ -2068,6 +2057,67 @@ export function getAgglomerateSkeleton(
20682057
);
20692058
}
20702059

2060+
export async function getAgglomeratesForSegmentsFromDatastore<T extends number | bigint>(
2061+
dataStoreUrl: string,
2062+
datasetId: APIDatasetId,
2063+
layerName: string,
2064+
mappingId: string,
2065+
segmentIds: Array<T>,
2066+
): Promise<Mapping> {
2067+
const segmentIdBuffer = serializeProtoListOfLong<T>(segmentIds);
2068+
const listArrayBuffer: ArrayBuffer = await doWithToken((token) =>
2069+
Request.receiveArraybuffer(
2070+
`${dataStoreUrl}/data/datasets/${datasetId.owningOrganization}/${datasetId.name}/layers/${layerName}/agglomerates/${mappingId}/agglomeratesForSegments?token=${token}`,
2071+
{
2072+
method: "POST",
2073+
body: segmentIdBuffer,
2074+
headers: {
2075+
"Content-Type": "application/octet-stream",
2076+
},
2077+
},
2078+
),
2079+
);
2080+
// Ensure that the values are bigint if the keys are bigint
2081+
const adaptToType = Utils.isBigInt(segmentIds[0])
2082+
? (el: NumberLike) => BigInt(el)
2083+
: (el: NumberLike) => el;
2084+
const keyValues = _.zip(segmentIds, parseProtoListOfLong(listArrayBuffer).map(adaptToType));
2085+
// @ts-ignore
2086+
return new Map(keyValues);
2087+
}
2088+
2089+
export async function getAgglomeratesForSegmentsFromTracingstore<T extends number | bigint>(
2090+
tracingStoreUrl: string,
2091+
tracingId: string,
2092+
segmentIds: Array<T>,
2093+
): Promise<Mapping> {
2094+
const segmentIdBuffer = serializeProtoListOfLong<T>(
2095+
// The tracing store expects the ids to be sorted
2096+
segmentIds.sort(<T extends NumberLike>(a: T, b: T) => Number(a - b)),
2097+
);
2098+
const listArrayBuffer: ArrayBuffer = await doWithToken((token) =>
2099+
Request.receiveArraybuffer(
2100+
`${tracingStoreUrl}/tracings/mapping/${tracingId}/agglomeratesForSegments?token=${token}`,
2101+
{
2102+
method: "POST",
2103+
body: segmentIdBuffer,
2104+
headers: {
2105+
"Content-Type": "application/octet-stream",
2106+
},
2107+
},
2108+
),
2109+
);
2110+
2111+
// Ensure that the values are bigint if the keys are bigint
2112+
const adaptToType = Utils.isBigInt(segmentIds[0])
2113+
? (el: NumberLike) => BigInt(el)
2114+
: (el: NumberLike) => el;
2115+
2116+
const keyValues = _.zip(segmentIds, parseProtoListOfLong(listArrayBuffer).map(adaptToType));
2117+
// @ts-ignore
2118+
return new Map(keyValues);
2119+
}
2120+
20712121
export function getEditableAgglomerateSkeleton(
20722122
tracingStoreUrl: string,
20732123
tracingId: string,
@@ -2228,18 +2278,24 @@ export async function getEdgesForAgglomerateMinCut(
22282278
tracingStoreUrl: string,
22292279
tracingId: string,
22302280
segmentsInfo: {
2231-
segmentId1: number;
2232-
segmentId2: number;
2281+
segmentId1: NumberLike;
2282+
segmentId2: NumberLike;
22332283
mag: Vector3;
2234-
agglomerateId: number;
2284+
agglomerateId: NumberLike;
22352285
editableMappingId: string;
22362286
},
22372287
): Promise<Array<MinCutTargetEdge>> {
22382288
return doWithToken((token) =>
22392289
Request.sendJSONReceiveJSON(
22402290
`${tracingStoreUrl}/tracings/volume/${tracingId}/agglomerateGraphMinCut?token=${token}`,
22412291
{
2242-
data: segmentsInfo,
2292+
data: {
2293+
...segmentsInfo,
2294+
// TODO: Proper 64 bit support (#6921)
2295+
segmentId1: Number(segmentsInfo.segmentId1),
2296+
segmentId2: Number(segmentsInfo.segmentId2),
2297+
agglomerateId: Number(segmentsInfo.agglomerateId),
2298+
},
22432299
},
22442300
),
22452301
);
@@ -2254,17 +2310,22 @@ export async function getNeighborsForAgglomerateNode(
22542310
tracingStoreUrl: string,
22552311
tracingId: string,
22562312
segmentInfo: {
2257-
segmentId: number;
2313+
segmentId: NumberLike;
22582314
mag: Vector3;
2259-
agglomerateId: number;
2315+
agglomerateId: NumberLike;
22602316
editableMappingId: string;
22612317
},
22622318
): Promise<NeighborInfo> {
22632319
return doWithToken((token) =>
22642320
Request.sendJSONReceiveJSON(
22652321
`${tracingStoreUrl}/tracings/volume/${tracingId}/agglomerateGraphNeighbors?token=${token}`,
22662322
{
2267-
data: segmentInfo,
2323+
data: {
2324+
...segmentInfo,
2325+
// TODO: Proper 64 bit support (#6921)
2326+
segmentId: Number(segmentInfo.segmentId),
2327+
agglomerateId: Number(segmentInfo.agglomerateId),
2328+
},
22682329
},
22692330
),
22702331
);

frontend/javascripts/libs/async/debounced_abortable_saga.ts

+5-2
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,9 @@ import { call, type Saga } from "oxalis/model/sagas/effect-generators";
22
import { buffers, Channel, channel, runSaga } from "redux-saga";
33
import { delay, race, take } from "redux-saga/effects";
44

5+
// biome-ignore lint/complexity/noBannedTypes: This is copied from redux-saga because it cannot be imported.
6+
type NotUndefined = {} | null;
7+
58
/*
69
* This function takes a saga and a debounce threshold
710
* and returns a function F that will trigger the given saga
@@ -15,7 +18,7 @@ import { delay, race, take } from "redux-saga/effects";
1518
* is slower than a standard _.debounce. Also see
1619
* debounced_abortable_saga.spec.ts for a small benchmark.
1720
*/
18-
export function createDebouncedAbortableCallable<T, C>(
21+
export function createDebouncedAbortableCallable<T extends NotUndefined, C>(
1922
fn: (param1: T) => Saga<void>,
2023
debounceThreshold: number,
2124
context: C,
@@ -56,7 +59,7 @@ export function createDebouncedAbortableParameterlessCallable<C>(
5659
};
5760
}
5861

59-
function* debouncedAbortableSagaRunner<T, C>(
62+
function* debouncedAbortableSagaRunner<T extends NotUndefined, C>(
6063
debounceThreshold: number,
6164
triggerChannel: Channel<T>,
6265
abortableFn: (param: T) => Saga<void>,

0 commit comments

Comments
 (0)