distributed crawler structure

distributed crawler structure