If you're looking for fun data sets for learning, for teaching, maybe a conference talk, or even if you're just really into them,
sports offers up a continuous stream of rich data that many people can relate to.
Yet accessing that data can be tricky.
Sometimes it's locked away in obscure file formats.
Other times the data exists, but without a clear API to access it.
On this episode, we talk about PySport, something of an awesome list of a wide range of libraries,
mostly but not all Python, for accessing a wide variety of sports data from the NFL, NBA, F1, and more.
We have Kuhn Vassen, the founder of PySport, to talk through some of the more popular projects.
This is Talk Python to Me, episode 416, recorded May 11th, 2023.
Welcome to Talk Python to Me, a weekly podcast on Python.
This is your host, Michael Kennedy.
Follow me on Mastodon, where I'm @mkennedy, and follow the podcast using @talkpython, both on fosstodon.org.
Be careful with impersonating accounts on other instances.
There are many.
Keep up with the show and listen to over seven years of past episodes at talkpython.fm.
We've started streaming most of our episodes live on YouTube.
Subscribe to our YouTube channel over at talkpython.fm/youtube to get notified about upcoming shows and be part of that episode.
This episode is brought to you by JetBrains, who encourage you to get work done with PyCharm.
Download your free trial of PyCharm Professional at talkpython.fm/done dash with dash PyCharm.
And it's brought to you by InfluxDB.
InfluxDB is the database purpose built for handling time series data at a massive scale for real-time analytics.
Try them for free at talkpython.fm/InfluxDB.
A quick announcement before we jump into the conversation around PySports.
We have over 240 hours of Python course video content at Talk Python.
And if you're watching that content on a mobile platform, phone, or tablet, the browser is definitely not the best experience.
For example, on iOS, it won't even auto-advance or start playing without your interaction.
We've had some mobile apps for our courses for a while now, but they have fallen a bit into disrepair for a couple of reasons, including App Store tyranny.
But over the past four months, we've completely reimagined and rewrote our mobile apps in a modern and beautiful platform.
And I'm super happy to announce that they are now available for Android and iOS on both phone and tablets in their respective app stores.
So please visit talkpython.fm/apps to see for yourself how beautiful and clean the new apps are and why I'm so excited about them.
Download them for free and even take a couple of our free courses included in there, as well as the paid ones that you might have gotten, of course.
Finally, if you're curious for a bit of a look behind the curtain about how and why we rewrote them, check out my personal site, mkennedy.codes, for the full story.
Thank you all for supporting our work.
Now, on to the show.
Kun, welcome to Talk Python to Me.
Yeah, thanks.
Thanks for having me.
Really cool to talk about this topic.
Yeah, I'm real excited to have you here.
You have quite a collection of libraries.
Now, up front, these are not all your libraries, right?
These are sort of kind of an awesome list of Python and even beyond Python, sports libraries, data sets, APIs, models, everything, right?
I think it's for people, it was quite hard to find open source packages.
And I tried to collect everything I could find and make it available for everyone to find what I need.
It sounds like a great mission.
And as people will see, there is a bunch of stuff that we'll get you to talk about.
So I noticed not absolutely every sport is covered there, but many of the popular sports are covered.
And if you're interested in sports, I think also if you're interested in just examples that connect with people, right?
Imagine you're a university professor and you don't want to use the New York City tax data one more time.
You want to say, well, maybe people are into soccer, American football or NBA, whatever it is, right?
Maybe you could come up with something more interesting.
F1, for example, right?
Yeah, definitely.
There's quite some cool data available to use in your courses.
Yeah, absolutely.
Yeah.
Also, if people are members of some kind of club or team, maybe they could use some of this to bring some cool visualizations or analysis to their own organization, right?
Yeah, that's also one of the things that PySport likes to encourage to use open source packages that are already available instead of building your own stuff, because that actually happens a lot.
So that's also a part of the mission of PySport to make people aware of what's already there and try to bring people together.
One of the big problems, not problems, it's an opportunity, but it's also a challenge of Python.
If you go to PyPI.org right now, there's 453,000 packages.
I didn't know the number, but that's quite a lot.
We're coming up on half a million.
And if your goal is to work with some specific data set or try to solve a certain type of problem, often the hardest part is figuring out, well, what library do I use?
Does it exist?
And if so, is it up to date and all of these things?
So having a list like this, a place that aggregates it and sorts it, filters it, super neat.
So really looking forward to talking to you about it.
Before we get to that, though, just give us a quick bit on your backstory.
How'd you get into programming in Python?
Yeah, this is an interesting story.
I think I started programming when I was, I think, around 12, when I get Lego Mindstorms.
Lego that you could also program.
My father gave me a visual basic book.
Yeah, I should just figure it out.
So that's where I started with programming.
And then during high school, I also did web development with PHP.
I'm not really sure at what age, but eventually I ended up with, I think, the first Dutch search engine.
They want to, yeah, they need a Python developer.
And I didn't really know Python, but that wasn't an issue.
So there I learned Python.
And from that point on, I, yeah, only, or mostly used Python.
Right on.
You're like, all right, forget this PHP stuff.
I'm going Python.
Yeah, well, to be honest, we, my company, we still use PHP.
Yeah, I'm sure.
Because, well, it works quite well.
The performance is okay.
I'm not really sure if I'm allowed to say it on this show, but it also had some advances.
Yeah, absolutely.
Well, I mean, almost every language has some, something it's particularly good at and reasons to keep using it.
And then also there's just tons of software that was written, you know, pick your language, written in that language.
And it still works well.
And there's plenty of reasons to just keep going with it, right?
Yeah.
But, yeah, Python is really my language of most interest and what I really use on my day-to-day work.
Excellent.
Yeah.
Yeah, cool.
What are you doing these days?
Are you still working at the search engine?
No, that was quite a while ago.
I also worked at a huge online marketing agency where we did run the software department.
We created tools to collect all the data from all kinds of different sources and make it available for the teams.
Right now I'm running my own company.
It's called TeamTV, where we provide all kinds of tools where we use video and data for, example, performance analysts, but also for highlight creation or live streaming.
Just to make sure that we try, at least we try to combine video and data in all possible ways within the support domain.
Yeah, that sounds like a really interesting thing to be working on.
Yeah, I started mostly on the video engineering part.
So we built quite some stuff ourselves there.
So from people uploading huge amounts of footage that we need to transcode and how to scale it, how to serve it.
So that's stuff we build ourselves.
Yeah, later on, we keep on building more stuff around data and always keep the combination between data and the video.
Because, well, you can see some sort of metric, but you always want to see the footage behind it to actually understand the context of it.
Yeah, sure.
Sounds really fun.
And you're also involved with PyData Endover, is that right?
Yeah, yeah.
Give them a shout out.
Yeah, yeah.
Five years ago, we started with PyData Eindhoven.
We were already friends with PyData Amsterdam.
They said, well, maybe you should also start an Eindhoven chapter.
I think this year will be the anniversary, the five-year anniversary for PyData Endover.
And it's, yeah, it's an amazing community.
And that's, yeah, also inspired me to start with PySport.
I'm not really sure if people at all that are listening to this podcast know PyData.
Maybe, I think.
I hope so.
Yeah, I suspect most of them probably do.
At least the data science inclined among us.
Yeah, I can tell a little bit about it.
Right now, we have a nice way of organizing the meetups and trying to get more people involved
and talk about data science and share knowledge.
And then once a year, we have the conference where we try to get, yeah, collect money that
we can send to NumFocus and they can share it over all the open source projects.
So, yeah.
It's a really amazing community.
That's excellent.
Yeah, NumFocus does a lot to support the bigger data science oriented projects.
Yeah.
I think that's kind of unique amongst the, in the Python space.
You know, there's not really anything like that in the web or UI.
You know, there's not a lot of areas where there's like organization that says, okay, we're going to try to find the popular projects and support them across organizations.
Right.
Like people support Flask, but they don't also support Django in the same sort of organization.
Right.
I think it's also an opportunity for all the companies that are using those open source packages to give back.
And I think doing it through NumFocus, it makes it also easier because they use a lot of packages and can just donate to NumFocus and they will make sure it's distributed over those packages.
Right.
Absolutely.
If you use Pandas, you should also support NumPy, right?
Because that's kind of the foundation and so on.
Yeah, yeah, yeah.
Interesting.
Oh, that makes a lot of sense.
All right.
Well, let's jump into sports and your project, PySport.
Are there other people who are maintainers and working on this or is this just your project?
To get the meetup that we had just a couple of weeks ago, we had some more people collected.
And now we are building from there on to get more people involved with just PySport.
But one of the projects we built with PySport is the Cloppy package.
And there we have worked together with Jan van Haaren.
He's a head of data science at Club Brugge, a big club in Belgium.
We had the main maintainers there, but I think right now we have 22 contributors to the package.
So there's quite some people contributing there.
Yeah.
Yeah, that's a big group.
That's a lot of people contributing.
Yeah.
Let's start it this way.
Tell people what PySport is and about that.
And then we can talk a little broadly just about sports analytics before we get into the details.
The most important mission of PySport is to bridge the gap between the clubs and the sports analytics and just people and by using open source packages.
Because a lot of clubs are using open source packages.
Open source packages are used by the clubs and people want to have a way to contribute to their favorite club.
I think a lot of people are still struggling on how to do it.
And with PySport, yeah, we want to share the knowledge and teach people on how to do it.
So we try to get the experts from the clubs, but also getting the knowledge from, you know, like Thundas or other big packages and see how we can get all that knowledge into the sports analytics community.
With Cloppy, we try to set an example on how to build such a package, how to work together on such a thing and also encourage people to contribute.
Yeah.
Show that you don't have to create a pull request that there's a major effector, but also like minor things like typing errors, fix and documentation and show people that that's also very valuable to a package.
Interesting.
So Cloppy is standardized soccer tracking and event data, right?
So you started out with soccer or as I guess a lot of the world might refer to it as football, but in the U.S., that's already taken.
It's a namespace collision.
Yeah.
It's sometimes, yeah, it's difficult to talk about football, but here in Europe, we call it football, but for the package, because it's international.
It's also worldwide, so I call it soccer.
Yeah.
Namespace.
So give us a quick bit of background on Cloppy, but since it's kind of one of the founding, you created this as a way to sort of set an example, right?
For how to create a package and it helps people understand this event, this club data.
Where that started is on Twitter, there was already quite some people talking about sports analytics, of course.
And one guy, Joe Mulberry, he's working at a Danish top club.
He asked for help because he created a notebook and he wanted to build a Flask API on top of it.
And I said, well, I know Python.
I don't know really much about soccer or about data, but yeah, I would like to be involved.
I would like to help you.
And when I received a notebook, I noticed that like 80% of the code was about reading and standardizing the data to a format that he could work with.
And when we talked about it, it seemed like most, at least a lot of people are struggling with that issue and doing the same thing over and over again.
Because in more notebooks that I saw, people were doing the same thing, but in different ways.
And some were not correct implementation or inefficient implementations.
So I thought, well, one thing I know is how to read data and how to get it into a standardized format, because that was also one of the things I did at an online marketing company.
Yeah.
Like, I don't know much about your data format, but I know about processing data and normalizing it and all that, right?
Yeah.
I built a package start with just tracking data, but also try to explain what the next steps could be.
And then people said, well, this is really useful.
And from that part, I kept on adding deserializers for different kinds of data for the tracking data and also for the event data.
Yeah.
It tried to get knowledge from non-sport bigger projects.
So I also got Will Kunnen from the Texel package.
He also did several reviews on this package and give feedback to try to get the package on a higher level.
So people within sports analytics community could also gain more knowledge from there.
But maybe also a small background on the data.
So the tracking data, that's like positioning data for all players on the pitch.
I think it's most of the time 25 frames per second.
So you know the location for each player and the ball.
And on the other side, you have the event data.
So there are all passes and shots and things like that.
At this time from this position, there was a shot on the goal or there was a pass or there was a takeaway or penalty.
Yeah.
That's event data.
Yeah.
And all the vendors choose different formats.
Yeah.
Oh, geez.
That sounds hard.
So first of all, 25 hertz of all the people's location.
This is beyond somebody with just a pen and paper and notebook writing down, oh, at this time there was a shot on the goal by number 25.
How do they get that data?
That's crazy.
Yeah.
That's quite an advanced systems that they use.
So in the stadium, I think they have like 20 cameras around the pitch.
They use computer vision to detect all the players and combine it.
But I believe, and I'm not really sure if they're already vendors on the market that do it totally automated.
But I think from the system that I'm currently used in soccer, there are still some people needed for difficult situations like a corner kick, where a lot of people in a small area and a lot of occlusions happen.
You can't see the numbers, yeah.
Yeah, they can't see the numbers.
So just after a corner, some manual operator has to reassign some players or correct something.
But it's quite an advanced system already.
It sounds incredibly advanced.
It sounds like an awesome data set to work with because with that much data, you really can make a lot of interesting predictions and trends.
I mean, at some point, maybe we'll just put some sort of tracking RFID thing on the back of the player's heads, just stitch it on there.
And then you can fully automate it, you know?
Yeah, I think a soccer day ticket.
Yeah, maybe.
Yeah.
Not sure if all players would accept it.
But for example, on ice hockey, yeah, you can put it on the helmets.
Yeah, you can put it on the helmets, sure.
For football.
Yeah.
Things like automobile racing, you know, they have, not all of them, but for example, F1 has incredibly high frequency of, like, points that measure where is this car, how fast is it going, the cars are sending out real-time telemetry.
There's certainly many sports that have quite high fidelity in their data.
I must admit, I haven't seen the data from F1 yet, but it would be really interesting to learn from them and now to work with data and see if it can be applied to football or soccer or the sport.
This portion of Talk Python to Me is brought to you by JetBrains, who encourage you to get work done with PyCharm.
PyCharm Professional is the complete IDE that supports all major Python workflows, including full-stack development.
That's front-end JavaScript, Python backend, and data support, as well as data science workflows with Jupyter.
PyCharm just works out of the box.
Some editors provide their functionality through piecemeal add-ins that you put together from a variety of sources.
PyCharm is ready to go from minute one.
And PyCharm thrives on complexity.
The biggest selling point for me personally is that PyCharm understands the code structure of my entire project, even across languages such as Python and SQL and HTML.
If you see your editor completing statements just because the word appears elsewhere in the file, but it's not actually relevant to that code block, that should make you really nervous.
I've been a happy paying customer of PyCharm for years.
Hardly a workday passes that I'm not deep inside PyCharm working on projects here at Talk Python.
What tool is more important to your productivity than your code editor?
You deserve one that works the best.
So download your free trial of PyCharm Professional today at talkpython.fm/done with PyCharm and get work done.
That link is in your podcast player show notes.
Thank you to PyCharm from JetBrains for sponsoring the show and keeping Talk Python going strong.
I bet it's a lot, actually.
I bet it is, you know, just in terms of actual quantity of data, you know, how fast are sampling and how many cars for how long.
It's probably a lot of data.
That's also one of the interesting things about working with sports data.
I think the data engineering part and this package just focused on reading the data.
But then the next step, yeah, how to work with the data, especially if you would like to use the tracking data for a whole season.
Yeah, that's quite some data that also vendors can start struggling a bit with.
It just occurred to me, there's probably a whole nother demographic or aspect who would be interested in this kind of data.
It would be like sports betting people.
I mean, not that I have any interest in that at all.
But if you were trying to figure out like, OK, if this team plays that team, if you can understand, OK, this their star player,
if we match up their moves against the other person's moves, it turns out there's a weakness in this way for their defense or who knows.
Right. I mean, there's there's with that much data.
There's probably some interesting stuff you can do.
I think that a lot of vendors of the data also have the, yeah, the betting industry as well as their clients.
Because, yeah.
I don't really care to work for them or support them.
It's a little bit shady, I suppose.
But it does seem like you could, it's almost like really detailed information about companies for the stock market.
This is kind of like a little bit like that for the sports betting in some ways, I suppose.
Yeah.
Interesting.
I think one of the challenges here is probably a lot of this data is not easily offered up.
There's probably not a lot of JSON APIs with low latency that are super easy to access or some there must be, but not.
There's probably a lot of data out there that is not overly welcome to either be given out or it's given out over in batch over slow periods or something like that.
Right.
Maybe speak to a little bit about the data availability.
Yeah, that's quite an issue.
And I know mostly about the soccer data, but I can imagine that the same applies to most of the other sports.
And I think data availability is a major issue, at least if you want to encourage the community to work with it and do research on it and get people build more cool stuff without being within a club.
There are some companies that already provide quite a big setup of open event data.
Statsbomb is one of them.
I think they provide around 1,500 data sets for event data.
But if you're looking at the tracking data, maybe there are like 10, maybe 15 sets available because all those vendors have deals with the leaks.
They are not allowed to start it.
So you have to know someone within a club.
Or use a beautiful soup or scrapey or something like that, right?
That's the other option.
But then it's still very hard to get the tracking data because I'm not sure if you can actually scrape it.
But that's one of the things that I noticed when working on the open source of Piesport's website.
There are really a lot of scrapers.
And I think that's an indication that there's an issue with data availability.
Yeah, it's not.
This plugs into the API, but this is a scraper.
Yeah.
I guess it's worth pointing out or throwing out a bit of word of caution.
Just because the website is publicly available and you can hit it with some kind of scraping tool.
That doesn't mean you legally can do stuff with the data.
You probably want to be pretty careful about that, right?
Yeah, because I think even when it's not explicitly mentioned, most of the times it's not allowed to scrape the data at all.
But also in soccer, there are quite some websites that are explicitly forbid it.
And yeah, so the packages are there.
And it's also a bit, I was thinking about should I include them or should I not include them?
Because they kind of encourage non-legal actions, but yeah, not really sure about it again.
Yeah, sure.
I can see the case for both sides of that.
But I just want to let people know, just be careful with what you do with the data.
It's one thing if it's an academic research project and it's just for my own interest or whatever.
Yeah, if you start scraping that entire website and trying to make money out of it, you should not do it.
Or find a way to do it legitimately, right?
But just don't sneak through.
Yeah.
All right.
Well, I think it might be fun to, let's talk through some of the packages you have here.
So if you go to PySport.org and there's a nav bar and on the left it says open source.
And if people click that, then they end up with a whole bunch of, you know, I'll open it just this way for a moment.
We can look at it and talk about it.
So if you just click on it, it actually, there's a delay as it downloads.
Yeah, there's still something I need to fix because, yeah, it's quite some packages.
I mean, this is not a complaint.
It's just, I don't know how many pages that is, but that's a really small scroll bar.
What I noticed that's pretty cool is you can go in and there's a filter that you all have and you can filter by your language.
Right now you have Haskell, Python and R and others.
And then you can pick by sports and then you can pick by type of thing, right?
So I filtered our discussion down to Python libraries just because, you know.
We have a single title.
Yeah.
And you could also pick amongst the different types of tools.
So we talked about the scrapers and probably to a lesser degree, the APIs, right?
The API clients, which is cool.
There are some in there.
They say, here's the API and we just built a strongly typed package rather than just doing straight rest, which is great.
But you also have models and calculators like for predicting things.
And then IO for file formats, visualization, open data and databases.
Right.
So I encourage people to rather than try to read the whole list, which is hundreds and hundreds of packages to, you know, filter down maybe to the sport you're interested in or a couple of sports or the type of tooling you're interested in.
Yeah.
I think filtering is a must, but maybe if you have plenty of time, you could just scroll and see what's interesting because it's still, I think, a very interesting list to see what's just what available and get inspiration.
Yeah.
It's quite a list.
Yeah.
So what's the sort here?
If I come here, how do I, how does this get sorted?
Like, is there any meaning to the order they appear?
Is it just when they were entered or?
It's a good question.
I also open source the data collection part of this website, but it's daily collected, at least to provide an update.
And I think, I must say, I think there's an order.
And when I added the packages, I think that's the order here.
But to be honest, this can be pretty random.
Excellent.
All right.
Excellent.
All right.
So here, I'll just sort of go through a couple of the scrapers here.
And we can maybe dive into one or two potentially.
So there's PyBall.
We'll just go through just to give people a sense, right?
Of the ones here, right?
So there's PyBall, which is a Python API.
Nice.
Wrapper for stats.nba.com with a focus on NBA and WNBA application.
That's pretty cool.
I don't know anything about stats.nba.com.
But it looks like, yeah, this is a whole website with all sorts of data.
It's got players, teams, leaders.
Looks great, actually.
I think quite some people are also using this package.
I think it's a mostly used package when working with basketball data.
And it's not that they use the API to get this data.
Yeah, you get quite a bit of data here.
You've got like the player, their team, their age, their total number of points scored.
A lot of stuff you can do to sort of compare them.
And yeah, that's great.
So if you're into basketball, I think it's a great start.
It's also quite actively maintained.
That's also one of the things that I intentionally mentioned on the list.
Because some packages are not really maintained well.
I think it's a benefit.
Yeah, one of the things in the list that you call out is the number of contributors,
the latest version, when the last commit was to the package.
That's pretty cool.
In the beginning, I thought, well, maybe I can just manually update the list.
But then I decided, I think data engineering is fun.
Let's find a way to automatically fetch the data and update it.
Also, the license is pretty important to show it here.
And also, I'm not going to commit to see how actively it's maintained, the latest versions.
And also the contributors.
Sure, the difference between a package with one contributor and one with 30 contributors.
That's a big difference.
It's a really big difference.
Yeah.
I think it's also good for people to see if there's a package with just a single contributor
that might give an opportunity to contribute to it or work together.
So PySport would like to encourage people to get involved in those projects.
Yeah, that's a good idea.
So that could help out here.
Yeah.
And each one of these packages, you can go in and open the details here.
And it gives you a little bit more information.
Like, for example, it actually lists the contributors and links to their GitHub profiles and choose
their website and the GitHub page and PyPI and so on.
Yeah.
And also, you can click on one of the contributes and see what other packages they built.
Oh, really?
Okay.
So, like, if I click on this one, yeah, they've done just this one.
Well, and this one, just a single one.
Yeah.
Some of them, they might have worked on multiple.
I know Dependipods worked on a few.
Yeah.
That's a really nice contributor.
Yeah.
The absolutely prolific open source contributor.
Yeah.
Works on my project too.
This portion of Talk Python to Me is brought to you by Influx Data, the makers of InfluxDB.
InfluxDB is a database purpose built for handling time series data at a massive scale for real-time
analytics.
Developers can ingest, store, and analyze all types of time series data, metrics, events,
and traces in a single platform.
So, dear listener, let me ask you a question.
How would boundless cardinality and lightning-fast SQL queries impact the way that you develop
real-time applications?
InfluxDB processes large time series data sets and provides low-latency SQL queries, making
it the go-to choice for developers building real-time applications and seeking crucial insights.
For developer efficiency, InfluxDB helps you create IoT, analytics, and cloud applications
using timestamped data rapidly and at scale.
It's designed to ingest billions of data points in real-time with unlimited cardinality.
InfluxDB streamlines building once and deploying across various products and environments from
the edge, on-premise, and to the cloud.
Try it for free at talkpython.fm/influxDB.
The link is in your podcast player show notes.
Thanks to Influx Data for supporting the show.
I didn't realize you could actually see all the projects that PySport knows about that that
particular user works on.
That's a cool aspect of it.
I spent quite some time on fetching all the data and trying to combine it.
Also, fetching data for PyTi and also do the similar for the R packages.
Yeah.
And seeing how to get all the available data on one place.
It also tries to fetch images or screenshots from the readmes of the repo storage.
That works for some.
Oh, yeah.
That's nice.
Screenshots are really going to be very helpful.
Less important on the scrapers, more on the visualizers, probably.
But still.
Yeah, definitely.
What is opensource.pySport.org written in?
It's written in React using Next.js.
So it was also quite an adventure for me because it's the first application that might also explain
why it's still a bit slow on loading because I didn't really dive into how to make it faster.
It used still WIND.
But in the backend, it's Python.
It's using Luigi.
Okay.
That's, I still think it's a pretty interesting tool because it's really simple to set up
like orchestration of some tasks.
Right.
Like the daily scraping, updating the packages and that kind of stuff.
Yeah.
And then there's a GitHub action that runs on a daily basis and then patches all the data
and updates and commits it in a different branch.
And that one gets deployed to the Purcell, I believe.
Okay.
Yeah.
Very interesting.
But if you are interested in the source, you can also, it's also HopeSource.
Okay.
Great.
So, highball for NBA.
We have the hockey scraper, which is for scraping NHL play-by-play and shift data with six contributors.
That's pretty interesting.
What you'll see on the filter list for every sport, there's a package also for the NHL, for
ice hockey.
That's a little bit less maintained, I think.
But I have to, I'm not really sure if it still works because with those scrapers, it can work
today and not tomorrow.
It doesn't even necessarily mean that they were intentionally blocked.
It could just be, hey, we've redesigned our site.
Doesn't it look awesome?
You're like, oh, the CSS selector is no longer pull up the thing.
So, yeah.
So, that's also on the scraping part.
If it's last commit is like a while ago, it might be broken.
Maybe, maybe not.
Yeah, sure.
All right.
Let's see some more.
I think the StatsBomb API is an official package.
It's also cool that StatsBomb provides an open source package for accessing their data.
Yeah.
What is StatsBomb?
I see that showing up in many places on these different packages.
Yeah.
StatsBomb is, I think, one of the leading providers of event data in football.
And I think in both football and soccer and in football.
So, they provide the event data.
So, everything that happens on the pitch, like passes, dribbles, interceptions, everything.
They are also one of the providers of the open data sets.
Okay.
Yeah.
They've got a free data section.
That's cool.
Yeah.
They proclaim themselves as data champions.
That's kind of cool.
Yeah.
I think the data is pretty good.
I think also one of the best in the market right now.
But at least that's what I heard from some users.
Sure.
They even have courses.
Modern scouting and data-driven recruitment.
That's kind of interesting, isn't it?
Yeah.
You also have to figure out how to apply data science in your job.
So, how to use it and how to use the data for scouting purposes.
Yeah.
If you work in a professional sports organization or even college sports, the U.S. at least,
there's a lot of recruiting people up from lower levels.
The tab is in all sports.
But I think the data is really helping to make the number of players that you have to watch
from the footage a lot less.
So, if you can already make a short list instead of watching 15,000 players, then it's really
convenient.
Sure.
Or maybe you're looking for a particular asset or a particular part of the play that a player
is good at.
Right?
Maybe you're looking for a quarterback for a football team that is especially good at running
the ball in addition to just throwing it.
Right?
You could ask the data for that and really narrow it quite quickly, I imagine.
And then you have to work with the data, figuring out how to extract it.
Because maybe that single metric that's really important for you is not available in the original
data set.
So, then you have to figure out how to work with the data and get those metrics out of
the raw data.
Yeah.
Maybe it's something calculated or inferred.
Yeah.
And that's also one of the things that happens in soccer based on the tracking data.
But it will probably happen also in football and all the other sports.
That clubs will define their own metrics based on, for example, tracking data.
And use that to figure out what players match their own play the most.
Cool.
Okay.
So, yeah.
That's what, as you can see, there's a bunch of stats bombs here.
Pi Baseball, an MLB game, seem to be a couple of things around baseball data.
And baseball is one of those games that's kind of, I feel like baseball is one of those games
that was almost created by a statistician just so they could come up with stats.
There's so many stats.
And, you know, people get averages, you know, that what kind of hitter are they?
Well, they're like a 0.3, you know, they're a 300 hitter.
What are I, you know, 30% and all that.
And I'm not a huge fan of baseball.
I find it kind of a slow game.
It's kind of fun to play, but to watch.
And it's like, you know, same as golf.
I don't watch those things.
Yeah.
I'm sure they're fun to play, but it's just like, in terms of stats, these kinds of games,
there's probably a ton of stats here because it's all about stats there.
I also believe that the baseball data science departments are one of the biggest departments
overall sport.
And maybe, but I'm not sure about it.
You can also make a lot of impact there.
Maybe.
Sure.
Because also in all the sport, for example, soccer, a lot of things has impact on the
eventual outcome.
It's also a discussion if all data is available to know what actually has the most impact.
So that's also one of the discussions within the soccer analyst community.
Yeah.
For both of these, Pi Baseball and MLB Game, you can see from your Luigi automation.
They're both quite, well, the MLB Game is not particularly up to date.
I guess the Pi Baseball one is more up to date.
But, you know, 13 contributors, 30 contributors.
That's quite a lot.
And the Pi Baseball was updated this month, right?
But, you know, when I saw these, I'm like, oh, these are kind of similar.
And then I look at your page here and I see, oh, well, Pi Baseball is, you know, way more
up to date, modern.
And you should check that out first, right?
That's the kind of value you get for having the info.
Yeah.
That's also the intention that you have a quite quick overview of, yeah, how it's maintained
it.
And, yeah.
Yeah.
And that one also goes against the API.
So let's see.
A couple more.
I guess it's worth giving a shout out to the NFL FastPie.
That, well, you know, NFL's got a lot of data as well.
What else?
There's some college baseball.
Here's one that I think is that shows up across a lot of the different categories because
it seems to do a lot, which is Fast F1.
Have you seen that?
Have you played with this any?
Also updated this month.
I should dig into it because quite some contributors.
And I think it's really interesting to also see the mode of sports or cycling or more of those
sports to see what they are doing, how they're doing it.
Yeah.
I noticed looking through here that there's not a lot of motor sports compared to the other
sports.
And so people, if you're out there, like if you're an IndyCar or if you're in motocross
or somewhere like, and you've got a package and shoot it over to these guys and have them
put it in the list.
That'd be cool.
Yeah.
The Fast F1, they've got a page here that has a bunch of things.
It has access to timing data, telemetry, session results, and all the data is provided
in an extended Panda data, Panda's data frame format, which is pretty cool.
Right.
Integration with Matplotlib.
There's an examples gallery too.
You come over here and you can see it has things like position changes during the race.
So this, it'll say, if you go up here, it'll do things like, you got to go forward, you
know, go to the C, get season 23 race one or for race, I guess, rather than practice or
qualifying.
And that's Bahrain.
And so then here's, you know, it has all the drivers, their time throughout the race, their
position.
You can see probably pit stop.
There's a lot of cool stuff you can see in here.
It looks really nice.
And also with those examples, I think it's really helpful to get people started with those
packages.
Yeah.
It's not exactly a Jupyter notebook.
It's the HTML of a Jupyter notebook.
But, you know, it's still exactly what you need, right?
But I think you can even download it by a notebook.
You download it right there.
Absolutely.
Yeah.
And apparently two and a half seconds to generate this script.
Let's see.
You can even got cool visualizations like on the track, color it by speed around the tracks
of the start.
You know, there's a lot of cool data here.
I'm not really sure why I haven't seen this one before, but yeah, it looks really, really
cool.
Yeah.
When I looked, I looked around a couple of the different packages and this one, like
the documentation and examples and stuff seem, seem super good.
Okay.
So that's the scrapers.
There's many more.
There's plenty more there.
Another one, models, calculators.
Maybe take us through some of the ones that stand out in this category.
Like, for example, there's Lori's code for Metrica tracking data.
I love it that it's just, it's Lori's code.
Good job, Lori.
Yeah.
So this is mostly about how to also do all kinds of modeling on top of it, do predictions
on top of data.
You know, one of the packages that I think is pretty interesting is the soccer action.
Yeah, of course.
Again, it's soccer.
There's only Python, possibly.
But for example, they have soccer XG, which is, what is that?
XG boost models for soccer event data?
That's the expected goals.
So what's the expected value for a certain shot?
If it should go in or not.
So it's also based on a position on the page, how many players are between the player with
the ball and the goal.
So you can use it to determine, yeah, how, if a player should score a goal and how many goals
he should make.
Yeah.
I think this is actually one of the really interesting aspects is the model and calculate it.
You know, the prediction side is pretty cool.
There's quite some work to do for PiSports because, for example, the expected goals.
There's also one of the things that I've seen in ice hockey.
Also, in other sports where you have to score within a goal.
And I think it would be cool to find a way to abstract it over all sports.
Yeah.
Because it is kind of the same idea, probably different data sets, but right.
Like scoring in hockey and scoring in soccer is from a structural perspective of the data
is kind of the same thing, even though it's really quite different in size of the goal and
how easy it is and all that.
Yeah.
But I think we can still learn from the other sports and see how they did it.
Yeah.
Train up a model, but on different data, right?
But same type of model, potentially.
Yeah.
Maybe some different features, but yeah.
Yeah.
So the next category is IO.
And that obviously stats bomb is in here, right?
Python package to parse, stats bombs, JSON data to CSV, which is cool.
Some on soccer, the spattle format, which I have no idea what that is.
Yeah.
That's also one of the things they built to make like atomic data format.
That's also kind of standardized.
So there's some overlap between soccer action and cloppy.
I think they mostly focused on how to eventually work with the data.
So calculate also the expected threat and also like a contribution model.
So for every action towards a goal.
Right.
Okay.
So maybe there's a takeaway and then a pass and a pass and then a score.
Like all of those people should somehow get credit for that potentially, right?
Yeah.
Okay.
Makes sense.
But they also build the way to load the data.
And they will currently also working together with them to see if we can make cloppy to load
the data and have the cloppy package focus on loading it and standardizing it and then have
the soccer action using it.
So see how the Nego blocks can work together.
Absolutely.
We have the NFLDB, a library to manage and update NFL data in a relational database.
That's kind of cool.
All right.
Let's see.
The next category is the visualization.
I think probably the most important part is probably the actual data acquisition, but
the most desired part is probably the visualization, right?
The data engineering part is not really, what do you call it, really sexy.
I mean, no one sees it.
The output is a structured CSV or packet file.
So that's not really cool to show.
But for example, the NPL soccer, I think it's a really, really nice package used by every
person in the soccer community.
Yeah.
There's a lot of contributors here.
Yeah.
And the visualizations look really cool.
Yeah.
They also have a huge list of examples.
Okay.
So all kinds of, you can just copy and paste to create some pizza charts.
I love them.
Yeah.
We'll actually come back to the pizza charts in just a moment, actually.
But yeah, these are some good looking visualizations here.
Yeah.
And I think the interesting thing about this package is that at some point there were two
packages that did similar things.
And then they decided, well, we should just work together.
And they spent quite some time on integrating those packages.
And then there was one.
And I think that's really cool to see that instead of kind of competing, they decided to
work together and make, I think, one of the most awesome.
packages for the soccer community.
It's really nice.
There's a lot of soccer ones in here.
Yeah.
There's also one for a PT plot for American football, although I don't understand what PT
stands for.
And then the fast formula one is also in there.
We already saw those pictures, but a lot of nice visualizations there.
Yeah.
And is that it for all the categories?
No.
Then there's the open data.
Yeah.
I think maybe when I look at this list, are some missing?
Okay.
It's still a bit limited on what data is available.
That's something that we should work together also with leaks to see if there's a way to make
some more data available.
Yeah.
They have it and they offer it publicly, put it in the list, right?
When it's available, I would definitely add it.
But there's already some interesting data.
There may be a little bit smaller data sets, but you can definitely use it to start playing
around with it.
All right.
So I think that kind of covers the list with the Python filter sort on.
You wanted also to give a quick shout out to NFLverse, right?
Because while not Python is quite a series of packages that does cool stuff in the NFL for
that data, right?
Yeah.
So it's not Python.
It's for our users.
But I think what's really interesting there, what they did is they created quite some different
packages, one for collecting the data, one for organizing it, one for reading the data,
one for doing all kinds of modeling, one for creating the visualizations.
And I think that's also an example for all the sports on how to make those packages available,
making sure that everything fits together.
Yeah, that's cool.
It's under the NFL virtual organization, but a bunch of different projects.
You know, you talked about having the data and stuff that's not immediately obvious or
predictable.
You might need a higher level sort of thinking about it.
And one of them that stands out here is the NFL fourth, which is studies fourth down decision
datas with the NFL version models, which is kind of cool because that's one of the big
decisions that a coach makes and it can make the game or it can lose the game.
And there's a go, no go decision, right?
And there's a lot of, it's not just, well, they went this far, then they didn't make it.
It's well, it was the, they had 30 seconds left in the game and they had to do it or, you know,
because otherwise they were just going to lose anyway.
Right.
There's a lot of higher, like sort of inference and higher level things you want to bring into
that rather than just 30% of the time they make it fourth down.
Right.
Yeah.
And this, I think also one of the reasons they just built an entire package around it
to work with it.
Yeah.
That's pretty interesting.
Now, before all the Python people say, I don't want to learn R, I don't care about R, it is
also worth pointing out that you can call R from Python.
I don't know how much like the visualization stuff still works super well or anything like
that, but you can use, or what is it called?
R pi two.
Okay.
And you can end up, you just pass it in our file and then you start calling functions or
whatever, get a, get a function out of it and call that function.
So it's worth, you know, if, if you really, really want to use some of these packages, maybe
it's worth doing a quick little integration and then turn it into a data frame, a pandas data
frame and running with it or something.
It looks interesting.
It's definitely worth a try.
It's nothing I've ever used, but I can see, you know, if you really care about NFL data
and you really care about Python, it might be worth, worth giving those, those combos a
look there.
I think there is one package to work with, with their data from Python.
So if you look at the list there, there should be at least one.
I think it's not on their website on their GitHub page, but I think there's another one
that integrates well with it.
Sure.
Right.
Not under the organization, but maybe somebody else.
Yeah.
That does.
Yeah.
That's cool.
Excellent.
Maybe they use this, this integration.
I don't know.
All right.
And then the last thing I want to talk about here is interesting on two levels.
So you've got a playground.
So you've got a playground.pysport.org, which is a hosted notebook to play with some examples,
like in particular, Cloppy and MPL soccer, right?
I think one of the issues or challenges for a lot of people also working within the bigger
clubs is that they don't always have a background in programming.
So often they start as a video analyst or working as a performance analyst, and then they think,
well, there's data.
I want to work with it.
And if you need to set up your Python environment for the first time, it can be a bit overwhelming.
So that's why I, for, well, there is JupyterLite, which is a very cool project based on Pyrodite.
Let's see if, yeah, if you can use it.
And it is just a start with the Cloppy and the MPL soccer package.
I just fetched the notebook from there, from my gallery, integrated into this one, into the
playground, and you can just start playing around with it.
Yeah.
And so here's a proper Jupyter notebook using all of their libraries and stuff.
But what's awesome about this, as you said, based on Pyrodite, I'm not sure it necessarily
actually stuck in people's minds.
Like, this is running in WebAssembly on our front end, right?
Which is pretty epic.
It makes it really convenient for people to just start playing around with it without installing
Python and working with virtual environments.
You know how it works.
Yeah.
It makes it super easy for you to host it because all you're doing is serving up static files.
You're not hosting, you're not running a Kubernetes cluster or anything like that, right?
Trying to prevent abuse of it and so on.
Yeah.
So, yeah.
Multiple sites make it good for me and for the people using it.
For sure.
And it even does that wild, what's it called, pizza plot, that kind of style of plot that
we're looking at.
And it runs fast and great.
Yeah.
This is really, really nice.
Yeah.
Are you happy with Pyrodite or Jupyter Lite?
Yeah.
There was some issues with it, especially around working with fetching data because some
of these try to fetch the open data from SlabsBomb or also some fonts and stuff like that.
Yeah.
So we had to work around it.
And it's also what you see on top of here is the patching of the request library to make
it work in Jupyter Lite.
Yeah.
I think it's better to have a working version than not patching it.
I think it's great.
And then everything that uses requests can just do its thing.
Yeah.
This is really cool.
When I saw that you had this, I thought, oh, this is clever that it's based on Jupyter
Lite.
And it's really nice.
Yeah.
So people can check that out.
Maybe people out there listening maintain some of these packages and have notebooks.
Like if they get them working here, could they submit them to you and have them added
in this list?
The entire playground is part of the PySupport organization on GitHub.
You can just watch, see the repository and make a pull request.
And I will just review it and merge it.
And then it will be available here.
Yeah.
That's awesome.
So I'm really happy for more packages here.
More examples.
Yeah.
More examples would be very welcome.
Excellent.
All right.
Well, I think we're getting pretty much short on time for talking about sports analytics,
but really, really good work there.
Now, before you get out of here, I have the final two questions for you.
I always ask.
Notable PyPI package, something you've come across.
You're like, oh, this library is awesome.
People should check it out.
I mean, it's kind of the whole topic of this show.
So we talked about, you know, maybe a hundred.
We didn't mention them all, but went through a list of a hundred different Python packages.
But something you want to give a shout out to that you think is cool out there?
I'm not really sure if the entire Python world already knows it.
But on the last PySupport meetup, I made an example using DuckDB.
That was something that people didn't know about it, especially with integration with Pandas data frames,
that you just build a data frame and run queries directly on top of it.
Yeah.
Interesting.
I heard a DuckDB, but I didn't realize the Pandas kind of direct integration.
It also has direct parquet.
Interesting.
Okay.
That makes it quite easy to also play around with SQL queries.
And I was very happy that I had a presentation on last PyData Eindhoven conference.
Yeah.
I think it's a package that, well, not everyone, but it's really worth checking out because it can make your life easier.
I think it's just a Swiss army knife for data engineering.
And yeah, I think it's a nice one.
Yeah.
Great recommendation.
And if you're going to write some Python code, what editor are you using these days?
I'm using PyCharm.
So I'm not, yeah, not sure if it's cool.
I love PyCharm.
PyCharm is awesome.
Okay.
Excellent one.
Yeah.
So I guess final call to action.
People are interested in open source sports analytics.
They're open and maybe interested in PySport, want to contribute back or, you know, be part of it in some way.
What do you tell them?
Yeah.
You can reach out on Twitter or LinkedIn to see where you can contribute.
And I think it's also, if you're not working in the support domain and would like to contribute, please reach out because I think the knowledge from outside of sports is really useful within sports.
So there are a lot of options to contribute and, yeah, make an even more community and make a more better community.
Yeah.
Absolutely.
All right.
Well, thank you so much for being here and sharing all these projects you've collected.
Thanks a lot for being on the show.
It's really, really nice.
Yeah.
Thank you.
You're welcome.
Bye.
This has been another episode of Talk Python to Me.
Thank you to our sponsors.
Be sure to check out what they're offering.
It really helps support the show.
The folks over at JetBrains encourage you to get work done with PyCharm.
PyCharm Professional understands complex projects across multiple languages and technologies,
so you can stay productive while you're writing Python code and other code like HTML or SQL.
Download your free trial at talkpython.fm/donewithpycharm.
Influx Data encourages you to try InfluxDB.
InfluxDB is a database purpose-built for handling time series data at a massive scale for real-time analytics.
Try it for free at talkpython.fm/InfluxDB.
Want to level up your Python?
We have one of the largest catalogs of Python video courses over at Talk Python.
Our content ranges from true beginners to deeply advanced topics like memory and async.
And best of all, there's not a subscription in sight.
Check it out for yourself at training.talkpython.fm.
Be sure to subscribe to the show.
Open your favorite podcast app and search for Python.
We should be right at the top.
You can also find the iTunes feed at /itunes, the Google Play feed at /play,
and the direct RSS feed at /rss on talkpython.fm.
We're live streaming most of our recordings these days.
If you want to be part of the show and have your comments featured on the air,
be sure to subscribe to our YouTube channel at talkpython.fm/youtube.
This is your host, Michael Kennedy.
Thanks so much for listening.
I really appreciate it.
Now get out there and write some Python code.
Thank you.