[Do Not Merge] Create memory root to avoid server error

[RRosio]

Bringing back this block of code that creates the root path for a memory file system. Currently, the root path is not recognised as a directory after we instantiate a memory filesystem. Maybe there are better ways to address this?
Follow-up from #86 and #87

[martindurant]

Can you please show what happens without this?

[RRosio]

Can you please show what happens without this?

Yes, sorry about that. Here is what I have now that I looked into it a bit more as well:

Browser Response Payload:

{"content": null, "status": "failed", "description": "/mytests2"}

*where the description is the error message

Browser Console Error Message:

index.ts:378 Uncaught (in promise) TypeError: Cannot use 'in' operator to search for 'status' in null at FsspecWidget.fetchAndDisplayFileInfo (index.ts:378:17) at async FsspecWidget.handleFilesystemClicked (index.ts:208:5)

Server output:

[E 2025-02-27 08:35:14.165 ServerApp] {
      "Host": "localhost:8888",
      "Accept": "*/*",
      "Referer": "http://localhost:8888/lab",
      "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36"
    }
[E 2025-02-27 08:35:14.165 ServerApp] 500 GET /jupyter_fsspec/files?key=[secret]&item_path=&type=default&1740674114160 (9549fc887bf14c79b22344953fed2e43@::1) 2.31ms referer=http://localhost:8888/lab

The issue is in my GET handler logic: https://github.com/fsspec/jupyter-fsspec/blob/main/jupyter_fsspec/handlers.py#L253
I check if a path is a directory or a file to call either ls() or cat(). This leads to calling cat() on the root path of the memory filesystem.
(I will also take this time to clean up that handler since I noticed there is redundant logic and also I need to have better error handling there)

[RRosio]

Below is the full output that I should see from the server.

Traceback (most recent call last):
  File "/Users/rosioreyes/miniconda3/envs/jupyter-fsspec/lib/python3.10/site-packages/fsspec/implementations/memory.py", line 231, in cat_file
    return bytes(self.store[path].getbuffer()[start:end])
KeyError: '/mytests2'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/rosioreyes/Desktop/code/jupyter/jupyter-fsspec/jupyter_fsspec/handlers.py", line 323, in get
    await fs_instance._cat(item_path)
  File "/Users/rosioreyes/miniconda3/envs/jupyter-fsspec/lib/python3.10/site-packages/fsspec/implementations/asyn_wrapper.py", line 27, in wrapper
    return await asyncio.to_thread(func, *args, **kwargs)
  File "/Users/rosioreyes/miniconda3/envs/jupyter-fsspec/lib/python3.10/asyncio/threads.py", line 25, in to_thread
    return await loop.run_in_executor(None, func_call)
  File "/Users/rosioreyes/miniconda3/envs/jupyter-fsspec/lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File "/Users/rosioreyes/miniconda3/envs/jupyter-fsspec/lib/python3.10/site-packages/fsspec/spec.py", line 884, in cat
    return self.cat_file(paths[0], **kwargs)
  File "/Users/rosioreyes/miniconda3/envs/jupyter-fsspec/lib/python3.10/site-packages/fsspec/implementations/memory.py", line 233, in cat_file
    raise FileNotFoundError(path) from e
FileNotFoundError: /mytests2
[E 2025-02-27 23:16:26.421 ServerApp] {
      "Host": "localhost:8889",
      "Accept": "*/*",
      "Referer": "http://localhost:8889/lab/workspaces/auto-p",
      "User-Agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/133.0.0.0 Safari/537.36"
    }
[E 2025-02-27 23:16:26.421 ServerApp] 500 GET /jupyter_fsspec/files?key=[secret]&item_path=&type=default&1740726986410 (331f2a38eec14765a60f0614b0006447@::1) 8.63ms referer=http://localhost:8889/lab/workspaces/auto-p

I think I can make these updates separately, so I can keep my changes here targeting to original issue. I intend to try updating and reusing the handle_exception functionality in

jupyter-fsspec/jupyter_fsspec/handlers.py

Lines 18 to 35 in c2690fb

	@contextmanager
	def handle_exception(
	handler,
	status_code=500,
	exceptions=(yaml.YAMLError, ValidationError, FileNotFoundError, PermissionError),
	default_msg="Error loading config file.",
	):
	try:
	yield
	except exceptions as e:
	error_message = f"{type(e).__name__}: {str(e)}" if str(e) else default_msg
	logger.error(error_message)
	handler.set_status(status_code)
	handler.write({"status": "failed", "description": error_message, "content": []})
	handler.finish()
	raise ConfigFileException

[martindurant]

This might happen on any filesystem - trying to list a directory that doesn't exist will be FileNotFound (if the filesystem has directories at all). So should generalise this to any protocol?

[RRosio]

Ah right, I don't think that this behavior aligns with typical filesystem expectations. Initially, I was considering cases where a user specifies a memory filesystem path in the config, where I might be inclined to expect the memory filesystem root to exist... but once users can add files and such to the memory filesystem, the root would eventually be created, so it's not really a problem.

I'm wondering if it would be best to propagate the FileNotFound to the frontend then, or would it be more appropriate to interpret the query to the root of an empty memory filesystem as an empty list? Since fsspec would normally raise an error, I'm leaning towards doing that for consistency. In terms of the user experience, I wonder if that could be misleading? I guess for that the backend can raise this error and let the frontend handle this case and how it's presented to the user? I would appreciate your thoughts on this!

[RRosio]

Since I'm leaning toward the first option, I think I may close this.... I'm working on the exception handling separately and I've pulled in the clean up commits from this PR there.