Guess the rating: adding more variables

Comments on https://lichess.org/@/oortcloud_o/blog/guess-the-rating-adding-more-variables/jVjb7EMf

First

@oortcloud_o Can you deploy the best performing model onto perhaps Hugging Face? Would like to try it. Preferably the dataset too, so I can train a custom Llama model based on it for better inference for testing myself for fun.

dboing

edited

Do you want such feedback in form of questions? they would help me getting further into your report and understand the hypotheses and what is shown exactly in the figures with respect to the full data set, and follow your statements per those figures. The figures themselves are clear if I assume i understand the previous aspect i am not sure about.. mostly precision on what you already provided. my intent is to help me understand and help you do that too, i.e. make me understand. As I do not doubt about the process you went through making sense.

I think my questions might be more about things you might assume known, and they might not be for some, including me (i might have some candidate, given my past experience with data analysis, but i would rather wait for your ok for me to post such questions here). Or it might be, if answered, about making emphasis in your explanations about things that might need to be, such as the split between training set and testing set (and related questions). also i do have some curiosity about aspect of your data sets that may not be explicit in the figures.

some of my questions may seem silly.. but they are genuine. others I am sure would be needed for not just me.

also sometimes using mathematical equation is of great precision help.. even if redundant with the text. like the error definitions.. I know digital ascii is math. adverse.. one needs to know that the rendering outlet would talk mathML, latex or even mere svg. I don't know how to solve that one.. don't you feel crippled by that?

Thank you for this report. I think i understand some of it. I do have a few questions about your presentation of the setup, which I notice you have described in more detail, and this helps understand what comes after. Do you want such feedback in form of questions? they would help me getting further into your report and understand the hypotheses and what is shown exactly in the figures with respect to the full data set, and follow your statements per those figures. The figures themselves are clear if I assume i understand the previous aspect i am not sure about.. mostly precision on what you already provided. my intent is to help me understand and help you do that too, i.e. make me understand. As I do not doubt about the process you went through making sense. I think my questions might be more about things you might assume known, and they might not be for some, including me (i might have some candidate, given my past experience with data analysis, but i would rather wait for your ok for me to post such questions here). Or it might be, if answered, about making emphasis in your explanations about things that might need to be, such as the split between training set and testing set (and related questions). also i do have some curiosity about aspect of your data sets that may not be explicit in the figures. some of my questions may seem silly.. but they are genuine. others I am sure would be needed for not just me. also sometimes using mathematical equation is of great precision help.. even if redundant with the text. like the error definitions.. I know digital ascii is math. adverse.. one needs to know that the rendering outlet would talk mathML, latex or even mere svg. I don't know how to solve that one.. don't you feel crippled by that?

dboing

edited

It might have been overkill. and you might start thinking seriously about how you split your training sets and test sets.

98% and 2% but how? maybe consider the old 10 parts cross-validation. This is not classification, but regression, but it should also work. The differences beijng the space of targets. not the sampling and sub-sampling story..

Since your focus in on comparing some classes of models, being more deliberate about how you measure that might be worth the analysis effort, and also the communication effort.. (they can help each other, i guess that is why you are sharing those reports, maybe not).

notions of underfit and overfitting arise and become clearer with more project attention on the data partitions, sampling and sub-sampling.. Which you have improved the sharing of, I should repeat that.

about the use of visual task optimized ML model. Here, pending full understanding of what is the setup, it appears that you are looking at correlations and cross-correlations, in a time dependent (or sequence of games, and rating trajectories), possibly tracking individual players, many of them.. Where among those would you expect convolution to find many 2D type of cross-correlations. It might have been overkill. and you might start thinking seriously about how you split your training sets and test sets. 98% and 2% but how? maybe consider the old 10 parts cross-validation. This is not classification, but regression, but it should also work. The differences beijng the space of targets. not the sampling and sub-sampling story.. Since your focus in on comparing some classes of models, being more deliberate about how you measure that might be worth the analysis effort, and also the communication effort.. (they can help each other, i guess that is why you are sharing those reports, maybe not). notions of underfit and overfitting arise and become clearer with more project attention on the data partitions, sampling and sub-sampling.. Which you have improved the sharing of, I should repeat that.

dboing

edited

I think i know my difficulty about the "setup" explanation (which is already a good improvement, me keeping on this is not meant to say otherwise, just keeping trying to understand).

My difficulty is that, in machine learning or what i call modern statistics, one is wanting to "extrapolate" from a training set to a test set (btw, both are supposed to be representative of the phenomenon under study where the whole sample of samples was taken (training union testing, in your case, since all i know is there is such a 2% excluded from training).

on one hand.. that is probably what you have assumed and the figure cloud are showing the test set and the predictor line or curve depending on model? I am focusing on your biggest model, which i can't quite figure the encoding of.

The art of encoding is crucial is such project. Go on medium seek articles with such key words. There is one with a bunch of them.. My point here, is that the sensivity of even the most flexible NN, on presenting the correct representation of what you intend to capture from the wild sample you are using, is big. Using a representation that might be very storage efficient, but far from making the useful information visible, is going to waste some of the NN training and function space flexibility toward unhiding the compressed stuff in its weight matrix space (some call that latent).

so using a vision optmized whole package, where I know the kind of encoding that might have been used at input layer would fit the big input vector you had to fill because of how the data came to you, and I did not quite get from the X and Y explanation,,,... and i finnish my sentence upon seeing i did not: so using .... is probably as you suspected the source (or necessary candidate to fix first) of the problem you share for that model experiment.

It becomes routine for people using NN, at some point what i just said. But i guess routine is often hidden from documentation, when audience is not specified, people write to themselves first. Not on medium (at least can find the right level for your state of familiarity, i have not much clue about that, only what i see in the blogs, which might be a handicap, having to use text for everything.

so. if you were you work on your X and Y explanantion. making sure to me more explicit about the Data Vector at input and the data vector at output (either training and testing) , maybe that is what was X and Y... you still need a paragraph for the data matrix (X and Y) elements ..

was is one sample of the dataset you used. how is the input element say little x itself probably a vector or index of event chronology or player id as you might have randomly assigned (or using lichess hashes, some ordering there).
and what is y. probably a single rating output.. that i could get.

i think the x from sample.. and then I would need to know the input encoding of that x at input layer.. maybe just the first distinctiohn between data things with uppercase, and the elements (the lichess individual samples) with lowercase. might help.

I hope i have not missed something obvious. that is quite possible.. in which case i apologize in advance. hopefully though it might help other readers interested by your report and like me feeling some misunderstanding remaining.

Keep up the data analysis questions.. I am not coder enough to induce what i just described as needed by inspecting your source code. Others might though.

I think i know my difficulty about the "setup" explanation (which is already a good improvement, me keeping on this is not meant to say otherwise, just keeping trying to understand). My difficulty is that, in machine learning or what i call modern statistics, one is wanting to "extrapolate" from a training set to a test set (btw, both are supposed to be representative of the phenomenon under study where the whole sample of samples was taken (training union testing, in your case, since all i know is there is such a 2% excluded from training). on one hand.. that is probably what you have assumed and the figure cloud are showing the test set and the predictor line or curve depending on model? I am focusing on your biggest model, which i can't quite figure the encoding of. The art of encoding is crucial is such project. Go on medium seek articles with such key words. There is one with a bunch of them.. My point here, is that the sensivity of even the most flexible NN, on presenting the correct representation of what you intend to capture from the wild sample you are using, is big. Using a representation that might be very storage efficient, but far from making the useful information visible, is going to waste some of the NN training and function space flexibility toward unhiding the compressed stuff in its weight matrix space (some call that latent). so using a vision optmized whole package, where I know the kind of encoding that might have been used at input layer would fit the big input vector you had to fill because of how the data came to you, and I did not quite get from the X and Y explanation,,,... and i finnish my sentence upon seeing i did not: so using .... is probably as you suspected the source (or necessary candidate to fix first) of the problem you share for that model experiment. It becomes routine for people using NN, at some point what i just said. But i guess routine is often hidden from documentation, when audience is not specified, people write to themselves first. Not on medium (at least can find the right level for your state of familiarity, i have not much clue about that, only what i see in the blogs, which might be a handicap, having to use text for everything. so. if you were you work on your X and Y explanantion. making sure to me more explicit about the Data Vector at input and the data vector at output (either training and testing) , maybe that is what was X and Y... you still need a paragraph for the data matrix (X and Y) elements .. was is one sample of the dataset you used. how is the input element say little x itself probably a vector or index of event chronology or player id as you might have randomly assigned (or using lichess hashes, some ordering there). and what is y. probably a single rating output.. that i could get. i think the x from sample.. and then I would need to know the input encoding of that x at input layer.. maybe just the first distinctiohn between data things with uppercase, and the elements (the lichess individual samples) with lowercase. might help. I hope i have not missed something obvious. that is quite possible.. in which case i apologize in advance. hopefully though it might help other readers interested by your report and like me feeling some misunderstanding remaining. Keep up the data analysis questions.. I am not coder enough to induce what i just described as needed by inspecting your source code. Others might though.