broadcast tensor or index

+1 vote
asked Apr 11 by siyuphysic (380 points)


I'm trying to use MPI to parallelize my codes. But I haven't found a proper way to broadcast a tensor or index among workers. Due to the complex structure of tensor in itensor, it's not easy to implement such tensor-broadcast functions efficiently.

Would someone share his/her idea on this issue?


1 Answer

+1 vote
answered Apr 11 by miles (16,570 points)

Hi Rui-Zhen,
I am actually working on an implementation of real-space parallel DMRG we hope to release publicly later this year. So that will have code inside it that uses MPI to send ITensors between nodes.

To help you out, I just put a file "parallel.h" into ITensor. So if you pull the latest version from GitHub you should see it in the itensor/util/ folder within the ITensor source code. Please treat the codes in there as somewhat experimental, although we have used them a lot in the past so they should be pretty well tested (we used them for the parallel DMRG paper).

The main object you want to use in parallel.h is the Environment object. You should immediately make this object at the top of your "main" function
Environment env(argc,argv);
(passing it argc and argv to create it). Then it will be destructed at the end of your program and will automatically call MPI_Finalize for you. The environment object "env" has a method env.broadcast(T) which takes any object T (such as an ITensor, IQTensor, SiteSet, MPS, MPO and many other types of objects or even plain data like integers) that can be read from or written to a binary stream.

MailBox is another type of object you can make to send to and receive from specific other nodes. You make a MailBox object and give it the id number of the other node (zero-indexed). then you can use the .send and .receive methods to do blocking sends and receives. I don't think it currently supports non-blocking sends and receives.

Finally, to give you more information about your question, every object in the ITensor Library has a .read and .write method that can read/write that object in binary to a stream (either an istream or an ostream). The codes in parallel.h create a stream (in some cases a stringstream, for example) and use the .read or .write methods of an ITensor (or another type of object) to read/write to this stream. Then the stream is sent to another node using a function provided by the MPI library.

So I hope the above paragraph helps you understand that you don't need to know about the "complex structure" of an ITensor or an IQTensor: you just have to use the .read and .write methods to convert an ITensor to or from a binary stream.


commented Apr 11 by siyuphysic (380 points)
edited Apr 11 by siyuphysic
Hi Miles,

Thanks a lot. I'll try your new codes today.

commented Apr 11 by siyuphysic (380 points)
edited Apr 11 by siyuphysic
Hi Miles,

"parallel.h" works very well. This is definitely what I want. It's a smart way to broadcast tensor via stringstream. I appreciate a lot.
There is a small mistake in 'parallel.h'. I think we should change and obj.write(data) to be read(data, obj) and write(data, obj).

And I'd like to share my simple example below. Hope this would help some people.

#include "itensor/all.h"
#include "itensor/util/parallel.h"

using namespace itensor;

main(int argc, char *argv[])
    Environment env(argc, argv);

    std::cout << "Worker " << env.rank() << " of " << env.nnodes() << std::endl;

    // Construct and print an ITensor
    auto a = Index("A",5);
    auto b = Index("B",4);

    auto T = ITensor(a,b);


    std::cout << "After broadcast:" << std::endl;

    return 0;
commented Apr 11 by miles (16,570 points)
Glad it's working well and thanks for posting the helpful example.

About using read(obj,data) etc. you are correct about that. The DoRead and DoWrite helpers in that file are older functions that precede the current read and write functions available in ITensor, so I'll change that.
Welcome to ITensor Support Q&A, where you can ask questions and receive answers from other members of the community.

Formatting Tips:
  • To format code, indent by four spaces
  • To format inline LaTeX, surround it by @@ on both sides
  • To format LaTeX on its own line, surround it by $$ above and below
  • For LaTeX, it may be necessary to backslash-escape underscore characters to obtain proper formatting. So for example writing \sum\_i to represent a sum over i.
If you cannot register due to firewall issues (e.g. you cannot see the capcha box) please email Miles Stoudenmire to ask for an account.

To report ITensor bugs, please use the issue tracker.