Hard ratelimiting when using API

Help
,
1

Hey there,

I’m trying to upload about 90 entries as markdown text files to one of my blogs using the Write.as API. The purpose is to upload my archive of translated works before I can start uploading new ones. For this, I am using the Go text importer that is listed as an example of the API usage. The README itself mentions the possibility of using wildcards to upload several anonymous posts (which I would prefer so I can review each before posting it publicly). As such, I run this command: go/bin/writeas-import -u actuallydael coll/poetry/*/*.md, and it can only upload around one or two posts before it gives me errors while publishing:

Publishing...Created post ... from coll/poetry/...
Reading file...OK
Publishing...error publishing coll/poetry/...: EOF
Reading file...OK
Publishing...error publishing coll/poetry/...: EOF
Reading file...OK
Publishing...error publishing coll/poetry/...: Request: Post "https://write.as/api/posts": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
Reading file...OK
Publishing...error publishing coll/poetry/...: Request: Post "https://write.as/api/posts": dial tcp 166.78.139.11:443: connect: connection refused
Reading file...OK

I thought to myself that I was being ratelimited, which I can kind of understand, although I am using a password to log in after all, so it didn’t make that much sense. The surprise came when I tried to open https://write.as on my web browser only to find that I couldn’t even access the page. The entire page, as well as my blogs were not accessible to me after being “ratelimited”, and only returned back to normal after several minutes. While I can understand the ratelimiting to some extent (even though once again, I am using an account), I don’t understand why the page would be blocked completely, including my own blogs, for several minutes after this. I haven’t labeled it as a bug because it may be intended behavior, but I really hope it’s not.

Are the errors while using the CLI supposed to happen? Is there ratelimiting on the API while using accounts? Why does the assumed ratelimit affect my capability of viewing my own blogs? Does it have to be so long?

Thank you.

2

An update: I tried to do this again manually, one document by one, to see if I would get hit by the (assumed) ratelimiting. I could upload 5 markdown documents just fine, until the 6th, when it gave me the same error as above. I am right now unable to access the write.as domain, with no idea of when I’ll be able to access it again (it’s usually around 5 minutes, I think?). This is getting in the way of uploading my archive practically. Otherwise, I would have to copy and paste text, which does not include formatting, and would take even more time. I don’t know how the ratelimiting would even be hit when doing this by hand and slowly, but it still did.

3

I have the same issue right now. With a wait time of 30 seconds between posts, I am able to push 33 posts through the API before I hit a stop, but without getting an error message from the API:

Publishing post on write.as blog 'jfriedrich'...
Post xz3psqvodlss0w0w successfully created!
Extracting post '31C3 Talks'...
Writing post to file posts/2014-12-31T21:36:24.000Z_31c3-talks.md
Publishing post on write.as blog 'jfriedrich'...
Publishing post FAILED. Response:

@matt is there a rate limit that we might run into? What are the rules to use the API? So and so many requests within a certain timeframe or leaving enough time between requests?

Thanks in advance :slight_smile:

4

Hey @jason, yes there’s some rate limiting we put in place earlier this year that applies when creating posts. But for situations like yours, we have a way to get around this with Application Keys (more info on this thread).

If you can confirm your Write.as username for me, either here or in a message to @support, I’ll unlock that feature for you!