Comments on https://lichess.org/@/oortcloud_o/blog/preparing-for-next-analysis/JH16NoJe
I like the sharing of the process dealing with data in its raw forms. And how one have to fight with nested hypotheses and factors.
I think "cheating" is one way to travel, as bringing a lot of attention, but that the methodology you expose, or the difficulties you share might be usable for other questions. In that sense I don't mind the reading about your project.
There is one funky question I have often seen, when the core belief is that there are cheaters under the bed, that there is an invisible mass of cheater in hiding behind the online chess-blob, so that whatever data argument one might come up with to quantify cheating probabilities per user per game over some duration (or game number, -- might be less of a time control variable unless we include questions of daily stamina affect play quality), there will always be the chess AND cheater genius that will be able to appear as non-cheater. I put "chess" AND cheater, because often the prowess in cause is that the cheater might be so chess aware of where a more is more of human nature than engine nature, where it might not affect the cheater objective (which is? remind me) and yet allow the game to keep going on, and only using the engine where it matters.
I do not know of engines that would give such information as where its scoring it crucial or not? Maybe having more engine internal pedagogy could make such a thing clear. In anycase, someone who is chess savvy enough to interpret engine output (not lichess on another tab , that is for sure....;) toward such prowess (and i admit it might not be easy to do while blundering fast, but less that the opponent) is likely to not need to cheat. However I might have the wrong assumptions of cheater objectives.
The travel time to move a piece needs to be added to an analysis. A bit like say if you walk 13 Km in 2 hours, while if you bike to reach the same destination, the distance is no longer the same nor the time. Bot measurements to reach a move and normal human responses to reach the move are not the same and need to be differentiated during an analysis. Measuring the difference between a bot response and a human response may make all the difference. Was the keyboard pressed or was it generated by a bot? How was that move made? My move confirmations are done by pressing the enter key on the keyboard and the pieces moved by the mouse. What's the speed settings of the mouse? It's not always the move, its the equipment used.
So speed or slowness to a move needs to be added to the analysis. The one we look at is the expected move. So it's speed reaction will have increased like a per move response. A piece that was not expected to move now has eye movement time to reach that area of the chessboard and the time to become aware of what that move did to the position and could do. It then needs to also add the time to go grab that mouse and piece to reach anew square. The Lichess compensation time has to be known for every move!
The future analysis needs to add measurements for time & distance to detect a cheated move. Detecting one cheated move should be the goal and proof enough that assistance was used. Example Keyboard was press to generate the hint for the move.
An in-depth profile statistics on the players is what will help create a cheat signature. Maybe chess sites already have some sort of cheat signature record list that get sorted by DNS locations. Maybe one area in the world at a give time has more cheats. At what time of day in your country does there seem to be more cheats? So resources could be concentrated like radar detection systems during those time periods in those areas.
Maybe a player is pressing something in the developer tool section (Ctl+Shift+I) or on the keyboard to generate the best move when they want. If that area of the web browser was detectable or the keyboard inputs monitored for activity, it might help detect cheaters too. Analysis of the game would then be complementary proof of best move or 2nd best move choices.
The location of where the mouse goes, like outside the web page is a good indication of cheating too.
The green dot turns gray when a player leaves the main game page. So maybe those games could get a new feature request of adding a tag in the pgn to ID games that the green dot turned grey on and off during the game. they could then be in databases where they could then be scrutinized for probably of occasional cheated moves !
Minor comment but please use APIs over web scraping. If you load whole webpages instead of using the API then it places additional (and wasteful) load on the server - especially generating the profile pages which are quite intensive.
Another variable to consider is the standard deviation of the time a player uses to think. For example: in a game, a human player sometimes uses very little time to think about obvious moves (capturing queens, dodging check, etc.), and at other times more time to analyze the position, think of a plan or idea, and so on. Therefore, the standard deviation will be high. However, if a player is cheating using a computer, his standard deviation will be low, since the computer tends to be more even in its use of time. Also, if the moves are being entered manually, the cheating player's response speed will rarely be less than 3 seconds, even in the opening. Of course, it will be necessary to analyze what percentage of standard deviation can be considered high or low. This method will help to filter out at least those cheaters who use the computer to make each move.
sorry op for the off-topic probability of the following:
@somethingpretentious said in #4:
> Minor comment but please use APIs over web scraping. If you load whole webpages instead of using the API then it places additional (and wasteful) load on the server - especially generating the profile pages which are quite intensive.
Perhaps if the API was having some complete example for people not primarily of devellopper background, the web scapping temptation might have some competition in it appeal.
You can get support on the API and how to use it in the discord server (linked on the homepage) and in my experience using an API is far less difficult than web scraping, from a developer skill perspective.
Yes discord, i keep forgetting about that world. Ok for myself in future need. I might look there. But, still. Examples that handle the accidents that one might encounter, might help people figure out if that is for them or not. I would not know what to ask, until I got into trouble. Not really good for planning on which tools to use, if the priority is get the data and work more on the data analysis tools that might not be requiring the same extent developper skills, or just enough to get the data and process it. I am saying that if lichess really wants more people to try the API. I might not be understanding the kind of web scraping, I was assuming some browser extension, and html dumping kind of tools, which would come with the browser GUI features confort.