我可以限制Mojo :: IOLoop的连接数吗？

Question

我正在使用Mojo :: UserAgent对Web API进行递归调用。

在某些时候，这失败了 - 我怀疑，因为API对来自一个客户端的呼叫数量有限制。

是否可以限制Mojo :: IOLoop的并发调用数量？

我希望能够限制或计算并等待。

这是进行调用的代码：

my $ua = Mojo::UserAgent->new();

my $loop = Mojo::IOLoop->singleton;
$loop->max_connections(100);
$loop->max_accepts(100);

my $url = Mojo::URL->new('https://outlook.office365.com/EWS/Exchange.asmx');
# authentication omitted

# start navigating the org tree
tree($ua, '[email protected]');

sub tree {
    my ($ua, $email) = @_;

    my $xml = $mt->vars(1)->render($template, { name =>  $email });

    my $tx = $ua->build_tx(POST => $url =>
           {
            'Content-Type' => 'text/xml', 
            'Accept-Encoding' => 'None' 
            }
            => $xml);
    my $p = $ua->start_p($tx);

    $p->then(sub {
         my $tx = shift;
         my $dom = $tx->res->dom;
         my $reports = $dom->find('DirectReports Mailbox EmailAddress');
         if ($reports->size) {
             my @reports = @{$reports->map(sub { shift->all_text })->to_array};
             for (@reports) {
             print $email, $_;
             tree($ua, $_)
             }
         }
         })
    ->catch(sub {
            my $err = shift;
            warn "Connection error: $err";
            warn join ' ', @_;
    });
}

$loop->start;

这是发布的模板：

<soap:Envelope xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
           xmlns:xsd="http://www.w3.org/2001/XMLSchema"
           xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/"
               xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types">
  <soap:Header>
    <t:RequestServerVersion Version="Exchange2013_SP1" />
  </soap:Header>
  <soap:Body>
    <ResolveNames xmlns="http://schemas.microsoft.com/exchange/services/2006/messages"
                  xmlns:t="http://schemas.microsoft.com/exchange/services/2006/types"
          SearchScope="ActiveDirectory"
          ContactDataShape="AllProperties"
                  ReturnFullContactData="true">
      <UnresolvedEntry><%= $name %></UnresolvedEntry>
    </ResolveNames>
  </soap:Body>
</soap:Envelope>

这将返回一个XML文件，其中包含所有直接报告的电子邮件，然后用于进一步递归。显然，不能发布。

Answer 1

听起来你想要Mojo::UserAgent::Role::Queued，但这里是整个模块：

package Mojo::UserAgent::Role::Queued;
use Mojo::Base '-role';
use Scalar::Util 'weaken';

our $VERSION = "1.10";
use Mojo::UserAgent::Role::Queued::Queue;

has max_active => sub { shift->max_connections };

around start => sub {
  my ($orig, $self, $tx, $cb) = @_;
  state $queue //= Mojo::UserAgent::Role::Queued::Queue->new(
    max_active => $self->max_active,
    callback   => sub { $self->$orig(@_); }
  );
  if ($cb) {
    $tx->on(finish => sub { $queue->tx_finish(); });
    $queue->on( queue_empty => sub { $self->emit('queue_empty') });
    $queue->enqueue([$tx, $cb]);
  }
  else {
    return $orig->($self, $tx);    # Blocking calls skip the queue
  }
};

它创建一个队列，知道它可以同时处理多少个连接。它包装了start，因此在连接数低于阈值之前它不会运行。

我通过计算上一时间段的连接来完成速率限制的类似操作。