Torrent organization and rendering questions

[FrenchGithubUser]

Hi,

I had a look at the live demo (which is really great, thanks for making one !). It seems like there is not much organization besides categories and tags. Are there any plans for making something similar to what [Gazelle]https://github.com/OPSnet/Gazelle) offers, or is this totally out of scope (as it would require a different database schema) ? I had the idea of building a tracker/site with rust and I'm wondering wether I should contribute to torrust or start something else.

Also, if I understand well, nuxt.js does some server-side rendering. May I ask why this has been chosen over client-side only ? It seems to me that client-side rendering is more lightweight on the server (and scales better). Just being curious about this :)

Thanks in advance for your answers !

[josecelano]

Hi,

I had a look at the live demo (which is really great, thanks for making one !).

Hi @FrenchGithubUser thank you!

I think it's very important to have one so people can easily take a look. On the other hand I'm also documenting here all the paints of running a service like this. I think that could also be useful for other users.

It seems like there is not much organization besides categories and tags.

Yes, that's true. Since I joined the project we have been focused on building a good code base (refactoring, adding tests, containerization, ...)

Are there any plans for making something similar to what [Gazelle]https://github.com/OPSnet/Gazelle) >offers, or is this totally out of scope (as it would require a different database schema) ?

We have plans to improve the Index but we are an small team. Having a fully features private index in out of the scope for now. I see the Index could be a good (reliable and easy-to-sue) public index.

I think, one of the things we need to improve in order to maybe build something like Gazelle in the future are the metrics from the tracker. Private index require a lot of metrics from the tracker. I'm working on that in the Tracker:

• Overhaul stats torrust-tracker#1327
• Alternatives to the Tracker REST API torrust-tracker#1307

Besides, as you mention I also think maybe a NoSQL database would fit better to store torrents in the Index.

I had the idea of building a tracker/site with rust and I'm wondering whether I should contribute to >torrust or start something else.

Cool! That's something I would also like to do. I think one of the things why I prefer working on the tracker is because is written in Rust :-). I've proposed to take a look at some web-frameworks in Rust:

#640

I think we could reuse code from the backend. And we don't have many features yet in the current Index. From my side, I would at least build a proof of concept. But as I said we are an small team.

If you build it, for sure I will be with an eye on it and you are more than welcome to share your ideas. I will be happy to contribute as much as I can. Torrust is a community-driven organization and we want to connect with other people also interested in developing the BitTorrent ecosystem.

Also, if I understand well, nuxt.js does some server-side rendering. May I ask why this has been chosen over client-side only ? It seems to me that client-side rendering is more lightweight on the server (and scales better). Just being curious about this :)

My fronted knowledge is very limited but as far as I know Nuxt support different rendering modes:

https://nuxt.com/docs/guide/concepts/rendering

And you can disable server-side rendering.

I don't remember the reason why @mickvandijke decided to use Nuxt. I have not done too much work on this repo. But probably the reason to enable server-side rendering was SEO. It makes sense for a public Index.

Thanks in advance for your answers !

[FrenchGithubUser]

I think, one of the things we need to improve in order to maybe build something like Gazelle in the future are the metrics from the tracker. Private index require a lot of metrics from the tracker.

That's indeed one of the things, but I was rather referring to linking torrents together with groups, artists, collections, production labels, etc.

Besides, as you mention I also think maybe a NoSQL database would fit better to store torrents in the Index.

Yes, or something like graphql. afaik, NoSQL is rather for unstructured data, which wouldn't be the case of a gazelle-like db, so idk if this is the good choice actually. I'm not very well educated on db performances, but I think that a regular sql db isn't the most performant for a tracker that has a high level of organization (but I might be wrong !).

These 2 last points are the most important for me to know if I should start looking into the code of torrust's backend, or start a separate project. Are there actual plans in the short-middle term to implement this, or not ?

Depending on how the tracker (torrust-tracker) is built, I could reuse it on a separate project and plug it into the backend of the new project (or any other tracker, as long as it's not too rigid)

I've proposed to take a look at some web-frameworks in Rust

The backend is already written in rust, and for the frontend, idk if using rust would be a very useful addition, maybe some WASM parts, but not sure of the real benefits for a site like this.

And you can disable server-side rendering.

👍

[josecelano]

I think, one of the things we need to improve in order to maybe build something like Gazelle in the future are the metrics from the tracker. Private index require a lot of metrics from the tracker.

That's indeed one of the things, but I was rather referring to linking torrents together with groups, artists, collections, production labels, etc.

No plans for that in the short-term. Basically because we don't have capacity to implement them. But we don't have anything against having those features. If you want to add those features to the Torrust Index that would be fine for us.

Our short-term plans are:

• To improve administration
• To improve self-management tasks

With our current capacity that would take for sure one year or more :-(.

Besides, as you mention I also think maybe a NoSQL database would fit better to store torrents in the Index.

Yes, or something like graphql. I'm not very well educated on db performances, but I think that a regular sql db isn't the most performant for a tracker that has a high level of organization (but I might be wrong !).

These 2 last points are the most important for me to know if I should start looking into the code of torrust's backend, or start a separate project. Are there actual plans in the short-middle term to implement this, or not ?

For the tracker yes, one of my priorities is to implement the new GraphQL API. For the Index we are not planning to make big changes for now. However It would be fine to add a new GraphQL API. That's what we are going to do in the Tracker.

Depending on how the tracker (torrust-tracker) is built, I could reuse it on a separate project and plug it into the backend of the new project (or any other tracker, as long as it's not too rigid)

I hope it's not too rigid. In fact, our strength is the flexibility. The Torrust Tracker is not the most efficient (see https://torrust.com/blog/benchmarking-the-torrust-bittorrent-tracker) but we want it to be the most flexible. We recent and near future tasks are all oriented to make it more flexible:

• Improve stats: Overhaul stats torrust-tracker#1327
• Refactor packages: Refactor packages torrust-tracker#753 to increase reusability.
• The new GraphQL API: Alternatives to the Tracker REST API torrust-tracker#1307

In fact, the initial design was based on separate services for the Tracker and the Index. Even if that has led to some performance issues in the integration. Se for example: torrust/torrust-index#565

Importing stats in the Index from the Tracker is one of the things I would like to know if we can improve with the GraphQL. Not sure yet, my feeling, without having made any research or proof of concept, is GraphQL is only an implementation detail to let the Index subscribe to Tracker events, but without a carefully designed events we could continue having performance/scalability problems.

I don't have experience in event-drive architectures or GraphQL. That's the beauty of open-source projects, you can learn a lot of things :-).

I've proposed to take a look at some web-frameworks in Rust

The backend is already written in rust, and for the frontend, idk if using rust would be a very useful addition, maybe some WASM parts, but not sure of the real benefits for a site like this.

Yest, it's maybe only I'm enjoying using Rust and I want to use it everywhere or maybe because I think frontend stuff has nowadays a lot of accidental complexity, cycles and many frequent changes without a real benefit.

I have also thinking is a standard webapp interface is still a good choice. I think a kind of AI model chat would be a cool thing, but at the end of the day, you would need a web interface to show the users the results.

And you can disable server-side rendering.

👍

[FrenchGithubUser]

Thanks a lot for your detailed answers. I think that considering the choices that have been made I will take the route of building my own site (back/front) instead of contributing to the torrust-index/gui. Also, the goal of my project is to learn rust with a big project. Contributing to an existing one might limit me with this.

I also think that the visions are not really the same in terms of organization of the content. I'll keep an eye on this project though, and might even use the torrust-tracker to chain it with the site !

Lastly, I think that using an SQL db for a highly orgnized torrent site is not the most performant as there are a lot of relations in between entities. Something closer to a graph would suit it well I think. I started looking into surrealdb, which seems very promising.

Anyways, having more options is nice for the torrenting landscape ! But I'd say that the organization really is a dealbreaker for me, and I'd like to be able to do it from scratch, also with how the db is structured.