Skip to content

Commit 2673a44

Browse files
d4l3kpytorchmergebot
authored andcommitted
[distributed] add PG APIs and general doc cleanups (pytorch#140853)
Doc updates: * This adds documentation for the object oriented ProcessGroup APIs that are being used in torchft as well as pytorch/rfcs#71 . * It also does some general cleanups to simplify the distributed.rst by using `:methods`. * It adds `__init__` definitions for the Stores * I've reordered things so the collective APIs are before the Store/PG apis Test plan: ``` lintrunner -a cd docs && sphinx-autobuild source build/ -j auto -WT --keep-going ``` Pull Request resolved: pytorch#140853 Approved by: https://github.com/kwen2501
1 parent 5b326d6 commit 2673a44

File tree

2 files changed

+201
-72
lines changed

2 files changed

+201
-72
lines changed

docs/source/distributed.rst

+35-26
Original file line numberDiff line numberDiff line change
@@ -329,32 +329,6 @@ a github issue or RFC if this is a use case that's blocking you.
329329

330330
--------------------------------------------------------------------------------
331331

332-
Distributed Key-Value Store
333-
---------------------------
334-
335-
The distributed package comes with a distributed key-value store, which can be
336-
used to share information between processes in the group as well as to
337-
initialize the distributed package in
338-
:func:`torch.distributed.init_process_group` (by explicitly creating the store
339-
as an alternative to specifying ``init_method``.) There are 3 choices for
340-
Key-Value Stores: :class:`~torch.distributed.TCPStore`,
341-
:class:`~torch.distributed.FileStore`, and :class:`~torch.distributed.HashStore`.
342-
343-
.. autoclass:: Store
344-
.. autoclass:: TCPStore
345-
.. autoclass:: HashStore
346-
.. autoclass:: FileStore
347-
.. autoclass:: PrefixStore
348-
349-
.. autofunction:: torch.distributed.Store.set
350-
.. autofunction:: torch.distributed.Store.get
351-
.. autofunction:: torch.distributed.Store.add
352-
.. autofunction:: torch.distributed.Store.compare_set
353-
.. autofunction:: torch.distributed.Store.wait
354-
.. autofunction:: torch.distributed.Store.num_keys
355-
.. autofunction:: torch.distributed.Store.delete_key
356-
.. autofunction:: torch.distributed.Store.set_timeout
357-
358332
Groups
359333
------
360334

@@ -386,6 +360,7 @@ distributed process group easily. :func:`~torch.distributed.device_mesh.init_dev
386360
used to create new DeviceMesh, with a mesh shape describing the device topology.
387361

388362
.. autoclass:: torch.distributed.device_mesh.DeviceMesh
363+
:members:
389364

390365
Point-to-point communication
391366
----------------------------
@@ -506,6 +481,7 @@ Collective functions
506481
.. autofunction:: monitored_barrier
507482

508483
.. autoclass:: Work
484+
:members:
509485

510486
.. autoclass:: ReduceOp
511487

@@ -516,6 +492,39 @@ Collective functions
516492

517493
:class:`~torch.distributed.ReduceOp` is recommended to use instead.
518494

495+
496+
Distributed Key-Value Store
497+
---------------------------
498+
499+
The distributed package comes with a distributed key-value store, which can be
500+
used to share information between processes in the group as well as to
501+
initialize the distributed package in
502+
:func:`torch.distributed.init_process_group` (by explicitly creating the store
503+
as an alternative to specifying ``init_method``.) There are 3 choices for
504+
Key-Value Stores: :class:`~torch.distributed.TCPStore`,
505+
:class:`~torch.distributed.FileStore`, and :class:`~torch.distributed.HashStore`.
506+
507+
.. autoclass:: Store
508+
:members:
509+
:special-members:
510+
511+
.. autoclass:: TCPStore
512+
:members:
513+
:special-members: __init__
514+
515+
.. autoclass:: HashStore
516+
:members:
517+
:special-members: __init__
518+
519+
.. autoclass:: FileStore
520+
:members:
521+
:special-members: __init__
522+
523+
.. autoclass:: PrefixStore
524+
:members:
525+
:special-members: __init__
526+
527+
519528
Profiling Collective Communication
520529
-----------------------------------------
521530

0 commit comments

Comments
 (0)