A partial archive of https://discourse-mediawiki.wmflabs.org as of Saturday May 21, 2022.

Edit conflicts by tool not detected

LucasWerkmeister

I’m working on a tool to add and remove categories from pages. When implementing the edit via API, I tried to do “the right thing” so that edit conflicts would be detected; however, when I tried it out just now, this doesn’t seem to work, and my tool will overwrite whatever changes were made between it fetching the page and sending the edited version, with no warning.

You can see the source code on GitHub; in a nutshell, the actions I use are:

  • action=query, prop=revisions, rvprop=ids|content|contentmodel|timestamp, rvslots=main, curtimestamp=1, to get the wikitext to edit as well as timestamps for use below
  • action=edit, basetimestamp=…, starttimestamp=…, where both timestamps come from the previous API request

I tested it by inserting a time.sleep(30) on line 60, between the two API requests, and then manually editing the page during that time. But no matter what I do to the Wikitext – touching almost every line or even completely blanking the page – the action=edit request will not raise an error, and instead overwrite the changes that happened in between. The oldrevid of the edit returned by the API also does not match the original revision ID of the edit that the bot edit is actually based on (corresponding to the basetimestamp argument), but rather the revision ID of the manual edit I made in between.

I know the timestamps I’m sending aren’t None or anything trivial like that – here they are printed:

{'basetimestamp': '2019-03-12T23:55:49Z', 'starttimestamp': '2019-03-12T23:55:49Z'}

(In this case they’re identical because of the “setup” edit that is made right before the actual test starts. At other times they’re one second apart, with basetimestamp < starttimestamp.)

I also checked the parameter names – I’m pretty sure I’m not misspelling them. And just one time I did get an edit conflict, though I don’t know what exactly I did – since I didn’t have all my windows at the same time, I don’t know in what order things happened. (But you can see that there’s no QuickCategories edit after this edit.) And yet, most of the time MediaWiki seems to be ignoring those timestamps.

There are some existing Phabricator tasks for failure to detect edit conflicts, but all of them sound hard to reproduce and most of them seem to involve specific conditions as well that aren’t in place here. (Also, all of them seem to concern interactive editing, not editing via the API.) This case, on the other hand, seems to be reliably reproducible, so I suspect the error lies with me and not with MediaWiki. Does anyone see something I’m doing wrong?

LucasWerkmeister

I thought that maybe replication lag might be the issue: it looks like when ApiEditPage loads the WikiPage to edit, it doesn’t specify the $load mode, and the default is to load from a replica, which might not have seen the conflicting edit yet. But according to Grafana, s3 replication lag wasn’t unusually high at the time of these requests (there were probably some fifteen to twenty seconds between the manual edit and the tool edit it should have conflicted with, and the highest replication lag recorded is 11 seconds), so I’m not sure if that’s really the issue.

However, in the process I also discovered that ApiEditPage is actually mostly a wrapper around EditPage (with several comments about how this is “kind of a hack” and “kind of sucks”, but okay, that’s how it is for now). It’s possible that EditPage’s interface (i. e. URL parameter handling) changed in some way that ApiEditPage doesn’t account for, rendering the edit conflict detection (partially?) ineffective. I’ll look some more into this later.

(Also, at some point I suppose this should move to Phabricator.)

LucasWerkmeister

Um. Wait. This EditPage bit certainly looks suspicious:

if ( !preg_match( '/^\d{14}$/', $this->edittime ) ) {
	$this->edittime = null;
}

$this->edittime comes from EditPage’s wpEdittime, which ApiEditPage copies from its $params['basetimestamp']. But the basetimestamp I logged above certainly isn’t just 14 digits, it’s ISO 8601 formatted. So is EditPage just discarding it because it doesn’t match the expected format?

I’ll check tomorrow if that’s really the timestamp I’m sending, and if yes, where exactly it’s coming from – if the action=query API is really giving me a timestamp that’s not suitable for action=edit, or if my Python code somehow accidentally passes it through datetime or something. But not tonight, it’s already way too late.


Edit: no, I think ApiBase is responsible for turning timestamp parameters into “TS_MW format”. Nevermind.

Tgr

The API converts date parameters to MediaWiki timestamps automatically, yeah. Note that edit conflict detection during normal page editing is based on storing the last revision ID, and the API uses timestamps instead for legacy reasons (see T34037 and T58849) so there is no guarantee they are equally robust.

The other thing I’d check is whether you are running the script or the API on a VM where the clock has drifted.

LucasWerkmeister

Thanks for those links, it’s good to know the context behind those timestamps. Perhaps I’ll submit a patch to add baserevid to ApiEditAction.

But it turns out the reason my test edit conflicts weren’t detected is very simple:

$this->isConflict = true;
if ( $this->section == 'new' ) {
	// ...
} elseif ( $this->section == ''
	&& Revision::userWasLastToEdit(
		DB_MASTER, $this->mTitle->getArticleID(),
		$user->getId(), $this->edittime
	)
) {
	# Suppress edit conflict with self, except for section edits where merging is required.
	wfDebug( __METHOD__ . ": Suppressing edit conflict, same user.\n" );
	$this->isConflict = false;
}

Suppress edit conflicts with self, except for section edits where merging is required.

If I make the conflicting edit as a different user, the conflict is detected correctly. So it’s not really a bug in my tool nor in MediaWiki, I was just testing it incorrectly.

Edit: I’ve filed T218460 to request not suppressing edit conflicts for tool edits.

LucasWerkmeister

Edit conflicts with the same user are ignored by MediaWiki. With a different user, everything works.