Greetings and welcome to the Carousel Analytics blog. In this post I am going to look at a simple question with a seemingly obvious answer: Is staff experience important? Or, more specifically, do teams with more experienced coaching staffs have more on court success than teams with less experienced staffs? Conventional wisdom would say that it is better to have a more experienced staff, but I haven’t seen much work on this topic so I’m going to take a look and see if conventional wisdom is correct.
Before I get started, I want to describe the data and explain how some of the variables are defined. Across all D1 men’s basketball teams, there is a wide range of how many individuals are on a coaching staff in any given year. Jokes about whether your current favorite team actually has a functioning Head Coach (HC) or not aside, you can usually count on there being one HC and some combination of three Associate Head Coaches (AHCs) and/or Assistant Coaches (ACs). There is usually a Director of Basketball Operations, and there is frequently a Director of Player Development or Director of Player Personnel. Then there is the Video Coordinator. Possibly a Recruiting Coordinator. Occasionally a Director of Analytics. Not to mention the Graduate Assistants. And of course, the Student Managers. While it can be difficult to determine exactly how many people are on a staff in any given year, my data shows that since the 2007-2008 season on average a D1 men’s basketball team has about 5-6 people on staff, with a range of 1 to 12(!) staff members. In order to account for the variability in staff size, I have only included HCs, AHCs, and ACs in this analysis.
Even after limiting the data to only include HCs, AHCs, and ACs I still have to trim the data a bit to make sure that all of the teams used have the same number of staff members. Typically, a staff will have four “core” members, though occasionally there will be 3-5 core staff members. Usually a smaller staff is due to a staff departure late in the offseason, but sometimes teams just have less than 4 core staff members (i’m looking at you Chicago State). To account for this, I have removed any teams that did not have exactly one HC and three AHCs or ACs. This left a sample size of 3997 teams since the 2007-2008 season.
I will describe how staff experience is determined in a moment, but first I want to cover how I will define success. Instead of using wins or win percentage, I am going use to each team’s efficiency margin (points scored per 100 possessions minus points allowed per 100 possessions) over the course of the entire season.
Instead of using a single definition for staff experience, I am going to use three gradually tightening definitions. For each of the three experience definitions, I calculate an experience value for each coach on a team’s staff for a given year. Staff experience for each definition is then calculated by taking the sum of all four staff members of a team for each season. Teams are then placed into groups of (roughly) equal range based on staff experience. Experience definitions and results are as follows:
The first experience definition, Total Experience, is how much coaching experience each coach has from the high school/AAU level and up, including experience as both a men’s and women’s coach.
Looking at the figure below, there is a clear trend for teams with more staff total experience to have a higher efficiency margin, indicating more success. The lowest staff total experience group (15-25 years), which would be a VERY inexperienced coaching staff, on average allowed more points than they scored. Teams averaged a negative efficiency margin until staff total experience ranged from 47-56 years. Efficiency margin continued increasing throughout all experience groups until the most experienced group, 119-129 years of total experience, which declined. Though it is difficult to say whether that decline holds much meaning due to its small sample size (n=24).
The second experience definition, D1 Men’s Experience (All titles), is defined as how much coaching experience each coach has at the NCAA Division 1 men’s basketball level. This does not include any HS/AAU, professional, or Olympic experience. It also does not include any experience coaching women’s teams at any level. This definition does include any experience in any roll at the D1 level, such as DOBO, DOPP, GA, VC, Student Manager, etc…
Overall, the same takeaway can be seen when looking at Staff D1 Men’s Experience (All titles) vs Efficiency Margin. Teams with less staff D1 Men’s experience are less successful across the board, with the exception of the most experienced group (106-115 years), which again had a very small sample size (n=14).
The final experience definition, D1 Men’s Experience (Core), is similar to Staff D1 Men’s Experience (All titles), but different in that it trims off support staff experience and only includes years as a ‘core’ member of a coaching staff (HC, AHC, or AC).
Using this definition of staff experience, there is still an increase in on-court success as staff experience increases. Teams with less than 24 years of Staff D1 Men’s Experience (Core) have a negative average efficiency margin. The group with 14-23 years of D1 Men’s Experience (Core), while inexperienced, still had a fairly large sample size (n=422) to show a negative efficiency margin. As with both of the other definitions of experience, on-court success increased with staff experience until the final experience group (104-113 years), which had an extremely small sample size (n=9).
Across all three definitions of experience it appears that teams with more staff experience have greater success. While the increase in success with increased staff experience might be relatively minor, to the naked eye you can clearly see a trend. Statistically there is a relationship as well. A one-way ANOVA test was performed for each of the experience groups versus efficiency margin. The results all showed a significant difference (p≤0.05) in the mean efficiency margin for each of the comparisons. This indicates that for each experience definition at least one of the experience groups is significantly different from at least one of the other experience groups.
I am not going to dig deeper into each of the three experience definitions, but I will look more closely at the Staff Experience (Core) definition. The chart below shows which experience groups within the Staff Experience (Core) definition are significantly different from each other at the p≤ 0.05 level. Of note, if the p-value is less than 0.001 it is coded as 0.001. All significant values are shown in red.
Big picture, it takes a large sample size to find significance between groups, but significance is there. Implying that increased staff experience is related to increased success, at least when measuring success using efficiency margin. Despite fairly high variance of success within each group and what appears to be a modest increase in success with staff experience, mean efficiency margin was significantly different between many of the groups.
Looking at the group with 74-83 years staff D1 experience (Core) in the chart above, you can see that there is a significant difference between that experience group and every experience group up through 54-63 years of experience, after which there is no significant difference. In practical terms, this would mean that a staff with 74-83 years of combined D1 Head Coach, Associate Head Coach, or Assistant Coach experience has significantly more success than staffs with less than 63 years of the same experience.
In the end it appears that more experienced staffs are more successful, but this needs to be interpreted in proper context. Staff experience alone cannot win games, that is obvious. There are some data points that would impact on-court success that aren’t included here. I haven’t taken into account player quality, strength of schedule, injuries, etc… However, it isn’t much of a stretch to think that a more experienced staff has an impact on many of the variables mentioned. A more experienced staff will probably have more inroads on the recruiting trail and pull in better players. In turn, the school can schedule stronger opponents due to the increased quality of their own team, which would impact strength of schedule. It is hard to think of how an experienced staff could decrease injuries, but I’m sure someone could come up with a connection if they tried.
There may not be much real-world value in comparing the experience groups as shown above, but the message is pretty clear: If you are an Athletic Director or Head Coach putting together a coaching staff, it is worthwhile to consider the amount of experience of each of your individual staff members.
Thanks for reading. There will be more posts in the future. This post alone leads to a few follow up questions: Is it more important to have an experienced HC or an experienced staff? Or, how much success does an experienced staff help provide? I probably won’t post content on a daily, or even weekly, basis. But there will be more. If you have questions or comments, reach out to me on twitter @carouselytics.