Bidirectional synchronization for your posts

Hi!

I wrote a small program that allows you to maintain your blog locally on your computer, and then synchronize it with WriteAs. The synchronization is bidirectional, so if you edit something using the web interface, it will also be reflected in your local copy.

I’m using this to write my blog in Obsidian, and then publish the changes to WriteAs.

I also added support for SnapAs for pictures. writeas-sync automatically uploads the referenced images to SnapAs and rewrites the links in the posts it uploads.

You can read the installation instructions here: GitHub - Cyberax/writeas-sync: Sync WriteAs posts

It’s probably full of bugs, so I’d appreciate feedback (either in this thread or as Github issues).

4 Likes

Just here to say THANK YOU for putting this together and for launching it. This should allow me to use Obsidian or iA Writer with write.as in a much more seamless way.

I’m sure I’ll have more questions, but my first two are:

  1. With the file naming structure, are future dates respected and therefore this sync service supports scheduling of posts?
  2. How are pinned posts handled?

EDIT: I think through testing, I’ve determined that pinned posts are ignored, which is fine, but would be good to include in the documentation.

Thanks! I’ll add pinned posts in a new release in a couple of days. The dates in the future should be supported, but I haven’t tested how they behave.

Awesome. I’ve already run into the item you were afraid of with the pagination. I synced a site that has a lot of posts, and it only brought in the most recent 10, which aligns with what is visible when you visit the site. Based on the API and some work I did initially to bring the content in, @matt may be able to help there (both for function and rate limiting avoidance).

I’ll keep kicking the tires and let you know what other things I find. This is so great.

Local .md files sync to write.as, am I dreaming?
:heart_eyes: :kissing_heart: :pray: :partying_face: :tada: :clap: :+1:
Get the picture?
Now comes the time to update all the posts and slugs in the necessary format, and be ready when brew support comes along. I can’t wait!
@matt should be :crown: you!

EDIT: This should be a sticky!

I gather the list is worked from the older to youngest post by date, and then alphabetically from Z to A?
What happens when you have 2 (or more) posts on the same day? Like:

2023-11-08-untangling-the-aws-ssm.md originally published at 11 AM in the morning and updated on 2023-11-09 at 8 AM in the morning.
2023-11-08-multitenancy-in-postgresql.md originally published at 7 AM in the morning and updated same day at 5:01 PM in the afternoon.
2023-11-08-benchmarking-go-and-c-async-await.md originally published at 5 PM in the afternoon.

Write.as would show these to the public taking into account the publishing date/time from the most recent published on top, as:
2023-11-08-benchmarking-go-and-c-async-await.md originally published at 5 PM
2023-11-08-untangling-the-aws-ssm.md originally published at 11 AM
2023-11-08-multitenancy-in-postgresql.md originally published at 7 AM

Yes, the date part is used for post ordering. The YYYY-MM-DD format ensures that the ordering is both chronological and alphabetic at the same time.

When a post is published, the creation date property is also set on the Write.As side, so the order remains the same locally and remotely.

When you edit a post (either locally or remotely) the date part stays the same.

You don’t need to update anything! writeas-sync works with any Write.As account, it’ll download and name everything automatically.

Careful with the “everything” word. From my tests it’s still being limited to whatever is on the first page. There’s gotta be a way around that, I believe. I haven’t dug into the documentation on the API, but if pagination is what’s forcing that behavior I think there’s a different way to achieve getting the full site for the initial sync (at least).

But really you’d want the sync to be able to access any file so you can make edits to old posts too.

1 Like

I’m worried syncing based on alphabet before publishing time may switch posts location on blogs. An “older post” may get “younger” after a sync due to the alphabet considered before publishing time.

Shouldn’t it be date → publishing time → alphabet instead of date → alphabet ?

The sync process sets the date of the post when it uploads it, so it’s perfectly safe to first create a post 2023-01-01-happy-new-year.md, publish it, and then create a 2022-01-01-this-was-a-fine-year.md and publish it. They will appear in Write.As in the correct order (2023 first).

There’s a small caveat: there is no time in the timestamp, so if you have several posts within one day, they might get re-ordered into the alphabetic order within that one day.

From a simple test, I don’t think this solution will work for scheduling posts. I created a test post dated with tomorrow’s date. I ran the sync command and it shows the 2 local posts and 1 remote post on that site (the other one is the first post I created and synced yesterday with this script). I don’t find any indication of the new 11/11 post on the write.as side of things.

It seems like it only pushes the post to the write.as CMS if the date has occurred, but I could be wrong. I’ll see if it just appears tomorrow, but doubtful since I don’t see it when I log into write.as like I would if I scheduled it for the future through the web UI.

I’ve just added support for pagination.

It does seem to work, but it looks like WriteAs rate limits are not very high, so it still might fail for deep blogs.

2 Likes

Maybe a thought, ability to upload, download, delete .md files ? Like one would FTP or WEBDAV or even SSH? One could then use existing software to sync properly slug-named-formatted .md files?
No idea about technical feasibility of this …

Thanks for this.

@matt How can the rate limit for this sync process be better managed? If I remember correctly there are two tiers of API key available. I think you “lent” me one of the higher tier to do my initial ingest of content for one or two of my sites because I was hitting stricter rate limits with the standard one. Would this sync script be one that users of it could request a key for that better support it?

@Cyberax since I didn’t have to store my API key as a token when setting up the script, is this just using one you’ve preprogrammed into the script? If Matt gives guidance, I’d assume you’d be willing to update so that users could store their own key as a token for the script, correct?

EDIT/UPDATE: The initial sync on one of my sites with the new pagination resulted in a rate limit threshold at page 3, so this is def something we’ll need some @help from Matt on for at least the initial sync for sites to be able to use this to maintain from that point forward with maybe the normal rate limits. I’d guess the threshold is somewhere in the 25-30 calls.

1 Like

writeas-sync can download public posts without any authentication. Uploading will require you to set the WRITEAS_PASS environment variable prior to running the script (or using the ‘–password’ command line argument).

I’ve added REALLY long delays for rate limit errors, so now a full sync takes several minutes. But it works.

I just came across this, and it seems like a great tool. Are you still working on it? Are there additional features being planned?

My Go is a little rusty, but I’d be glad to pitch in if I could.

The code still works. The problem is in WriteAs, the rate limits are insanely low. I also looked at the WriteAs source code, and it’s a total mess.

I kinda lost confidence that WriteAs is sustainable. I’m still paying for a WriteAs account, but I’ll probably discontinue it sooner rather than later.