Wikipedia:Bots/Requests for approval/C1MM-bot 3

The following discussion is an archived debate. Please do not modify it. To request review of this BRFA, please start a new section at Wikipedia:Bots/Noticeboard. The result of the discussion was

Approved.

C1MM-bot 3

Operator: C1MM (talk · contribs · SUL · edit count · logs · page moves · block log · rights log · ANI search)

Time filed: 04:42, Thursday, December 12, 2024 (UTC)

Automatic, Supervised, or Manual: automatic

Programming language(s): Python

Source code available:

Function overview: Adds or modifies election templates in 'Results' section of Indian Lok Sabha/Assembly constituencies

Links to relevant discussions (where appropriate):

Edit period(s): One time run on a category of pages.

Estimated number of pages affected: ~4000

Exclusion compliant (Yes/No): No

Already has a bot flag (Yes/No): Yes

Function details: This bot modifies the results sections of Indian Lok Sabha/assembly constituencies. It takes the 'Results' section and for the most recent two elections with published data it adds in all candidates with vote percentages above 0.9% and removing candidates with vote percentages under 0.9%. It does not edit candidate data (i.e. hyperlinks are preserved) except to correctly capitalise candidate names in all upper case. 'change' parameter is only filled if there is no elections which take place between the two data.

Candidates are sorted by vote totals and the subsections are sorted by election years in descending order (most recent election comes first). If a 'Results' section does not exist, it is placed in front of the 'References' section and the results from the two most recent elections are placed there.

Discussion

What is the source of the election data being used by the bot? – DreamRimmer (talk) 14:27, 13 December 2024 (UTC)[reply]

The ECI website: eci.gov.in (it is geoblocked for users outside India). It has reports for every Parliamentary and Assembly election in India since Independence, and the ones after 2015 are in PDF form and those after 2019 have csv files. C1MM (talk) 01:19, 14 December 2024 (UTC)[reply]

Thanks for the response. I have used data from eci.gov.in for my bot task, and it is a good source. I tried searching for results data for recent elections, but I only found PDFs and XLSX files; I did not find any CSV files containing the full candidate results data. Perhaps I missed some steps. I will try to provide some feedback after reviewing the edits if this goes for a trial. – DreamRimmer (talk) 09:56, 14 December 2024 (UTC)[reply]

I convert XLSX to CSV (it is second-nature to do it now for me so I forget to tell sometimes). C1MM (talk) 17:07, 14 December 2024 (UTC)[reply]

Thanks for the response. Is the source code for this publicly available somewhere if I want to take a look at it? – DreamRimmer (talk) 09:44, 16 December 2024 (UTC)[reply]

There might be good reasons to keep a candidate's data even if they get less than 0.9% of the vote. I'd say that if the candidate's name is wikilinked (not a red link), then the bot should not remove that row.

Also, consider "None of the above" as a special case, and always add/keep that data when it is available. -MPGuy2824 (talk) 10:07, 14 December 2024 (UTC)[reply]

Good point. I forgot to mention I did treat 'None of the above' as a special case, don't cut it and in fact add it in where it is not in the template. I also add 'majority' and 'turnout' and when there is no election in between the two most recent elections for which I have data I also add a 'gain' or 'hold' template.

How do you check if a page exists and is not a disambigution? I say this because a lot of politicians in India share names with other people (example Anirudh Singh) so I would rather only keep people below 0.9% of the vote if they are linked to an article which is actually about them. C1MM (talk) 13:47, 14 December 2024 (UTC)[reply]

If you are using Pywikibot, you can use the page.BasePage class methods, such as the exists() method, to check whether a wikilinked page exists on the wiki. It returns a boolean value True if the page exists on the wiki. To check whether this page is a disambiguation page, you can use the isDisambig() method, which returns True if the page is a disambiguation page, and False otherwise. – DreamRimmer (talk) 17:07, 16 December 2024 (UTC)[reply]

I've made the suggested changes and the pages produced look good (I haven't saved obviously). I unfortunately don't know how to run Python pywikibot source code on Wikimedia in a way that accesses files on my local machine, is this possible? C1MM (talk) 05:56, 23 December 2024 (UTC)[reply]

Are you saying that you have stored CSV files on your local machine and want to extract the result data from them? Let me know if you need any help with the source code. – DreamRimmer (talk) 11:04, 23 December 2024 (UTC)[reply]

I figured this problem out. I would now think a BAG member should probably come and give their opinion. C1MM (talk) 16:56, 30 December 2024 (UTC)[reply]

{{BAG assistance needed}} — Preceding unsigned comment added by C1MM (talk • contribs) 16:55, 30 December 2024 (UTC)[reply]

Approved for trial (50 edits). Please provide a link to the relevant contributions and/or diffs when the trial is complete. Please do not mark these edits as minor. Primefac (talk) 13:34, 1 January 2025 (UTC)[reply]

[1] Here are the contributions asked for. I think there are a couple of issues: I haven't actually added a source technically for these contributions and also for a certain party (Peace Party) I added the disambiguation links by mistake. I also accidentally made the replacement headings 3rd level instead of 2nd level, which I have now fixed. C1MM (talk) 03:47, 2 January 2025 (UTC)[reply]

Please also go back and manually fix these 50 edits for the problems that you've noticed. Additionally, if you could also use the {{formatnum}} template for all the votes figures it would be great. The other parts of the edits look good. -MPGuy2824 (talk) 05:05, 2 January 2025 (UTC)[reply]

I've done what was asked. C1MM (talk) 04:33, 10 January 2025 (UTC)[reply]

I think you need to use the {{Bot trial complete}} template to bring this to the attention of somebody from the BAG. -MPGuy2824 (talk) 05:07, 10 January 2025 (UTC)[reply]

Trial complete. Noticed this went stale after completion. JarJarInks (talk) 19:21, 23 January 2025 (UTC)[reply]

Approved for extended trial (50 edits). Please provide a link to the relevant contributions and/or diffs when the trial is complete. Please fix the code to avoid repeating the same issues. Also, please link this BRFA in the edit summary and do not mark these edits as minor. – DreamRimmer (talk) 12:23, 29 January 2025 (UTC)[reply]

Courtesy tag @C1MM JarJarInks^{Tones essay} 21:46, 1 February 2025 (UTC)[reply]

Thanks, unfortunately I am busy in real life and don't have the time to work on this much at the moment. I am trying to add functionality to automatically add the correct references for templates I add which currently don't have references. C1MM (talk) 04:45, 3 March 2025 (UTC)[reply]

If you're able to open source I'd be willing to debug and add that functionality. Then you could come back and do the trial when you have the time. JarJarInks^{Tones essay} 14:11, 3 March 2025 (UTC)[reply]

Trial complete. Special:Contributions/C1MM-bot last 50 edits. I need to fix references: sometimes reference name is duplicated and sometimes it has an accidental "ref name = ref name =", so that needs to be fixed. Unfortunately open sourcing is a little difficult, although definitely not impossible. I have several previous files I need to run to get the election data into a wikitemplate format. One to save candidate information from PDF in a CSV, another to process CSV data, another to combine the PDF with CSV data so that vote percent changes are correctly added and finally the one which edits the current Wikipedia page by filling in missing information and reformatting. I would appreciate any help with getting this open sourced although I am still busy in real life. C1MM (talk) 05:17, 24 March 2025 (UTC)[reply]

@C1MM I unfortunately have a wrist injury and won't be able to do too much, but if you batch upload the files to a cloud service, and email me the link, I can try and link it all together cohesively. What environment do you typically run in? JarJarInks_{٩(◕_◕)۶}^{Tones essay} 12:00, 24 March 2025 (UTC)[reply]

I wanted to check if you had fixed the code for the issues from the first trial. However, you made extended trial edits to the same pages as the initial trial. Since those pages had already been manually fixed, there are no changes to review. I apologies if my extended trial approval was unclear; I intended for you to edit different pages so that we could verify whether the code has been fixed for the original issues. Also, we requested that these edits not be marked as minor, yet the code was not updated to prevent this. You also mentioned encountering problems, such as occasional duplication of reference names and instances of accidental 'ref name ='. Have you addressed these problems? I am giving you another extended trial. For this trial, please edit pages different from those edited during the first two trials. Ensure these edits are not marked as minor. Also, please resolve the reference name duplication and any other code issues before proceeding with these edits.

Approved for extended trial (50 edits). Please provide a link to the relevant contributions and/or diffs when the trial is complete. – DreamRimmer (talk) 17:18, 21 April 2025 (UTC)[reply]

Thank you very much for your patience. Unfortunately I have been very busy in real life but I have fixed the referencing issues and this should be ready to use now. I just completed the 50 edit trial which is available at Special:Contributions/C1MM-bot. C1MM (talk) 04:53, 9 May 2025 (UTC)[reply]

I checked a few of the latest edits at random and found a couple of minor issues:

In [2] a "-->" seems to have been added after the 2022 section title.
The insertion of the formatnum template isn't always done for the votes param e.g. [3]
Looks like the ECI results pdf is wrong on quite a few of the percentage numbers, but your script gets it right. e.g. [4]

Everything else seemed good in the smattering of edits that I checked. -MPGuy2824 (talk) 05:17, 9 May 2025 (UTC)[reply]

The --> was present before I edited the page, it was part of an incomplete comment.

I only add the formatnum template for templates I edit substantially, I don't want to change too much of templates which I don't have a source for.

As for the last point, I noticed that too, and I think it is only when the PDF numbers agree with the correct numbers that you see the script get it correct. I changed the script to calculate the percentages manually when processing the results PDFs. C1MM (talk) 03:13, 12 May 2025 (UTC)[reply]

Despite being told multiple times, you have still marked all edits as minor. This gives the impression that you are not reading BAG comments before making changes. – DreamRimmer ◆ 03:34, 12 May 2025 (UTC)[reply]

Ah, I realize the issue now. Very sorry about that: I didn't realize they had changed how minor edits were displayed recently. I've fixed the issue in my script so that the edits are no longer marked as minor. C1MM (talk) 13:00, 15 May 2025 (UTC)[reply]

Looks like this should have been marked

Trial complete. * Pppery * _{it has begun...} 16:35, 1 July 2025 (UTC)[reply]

Approved. Thanks for your patience. I hope you have fixed the code for the issues mentioned above. The edits look good. Please keep it under some supervision rather than making it fully automatic, as the trails suggest it may occasionally make mistakes where template or section formatting is slightly different. It would be good to keep an eye on the edits. – DreamRimmer ■ 06:33, 8 July 2025 (UTC)[reply]

The above discussion is preserved as an archive of the debate. Please do not modify it. To request review of this BRFA, please start a new section at Wikipedia:Bots/Noticeboard.