My talk from DSCC 2012
At DSCC 2012 I decided to go sans slides and just reflect on some of the issues in my field and topics that I think need to grow and change. I think the talk went over well. I at least had some folks show interest in sharing their work more. These are my notes that I drew the talk from.
- Introduction: Name, place, project, website
- Open Loop Plant model validity, Whipple deficient, motorcycle more data, tires, rider, more data
- statistical significance, ashamed even of me
- the human controller, most complex, link to handling, controls is too narrow, we need collaboration with other fields
- embrace open science for reproducible, reusable, rapid research
- data: make single track problem into big data problem, public repos, data papers, will allow us to rapidly generate models and validate models
- environing validating against hundreds of data sets
- data is cheap
- software, if we open data, same goes for software, code should be just as open as the equations in our papers, we need to know if gives correct results and we need the ability to hack it,
- meta software project that contains code for working with data, computing standard functions, models, controllers
- references, still manageable
- reproducibility: can we actually reproduce old experiments with modern methods?
- Meta papers, more review papers to put the previous research findings into perspective
Future of Bicycle/Motorcycle Dynamics
Hi, I'm Jason Moore from UC Davis. I've been working on bicycle dynamics problems for quite a few years now. I've spent most of my time trying to create models of the bicycle/rider system from experimental data. In particular understanding the system open loop plant and the closed loop control by the rider. I'm not going to go into much detail about my research today, it can be found at biosport.ucdavis.edu if you are interested in the research findings. I want to spend my ten minutes discussing some of the ideas and thoughts I've gathered over the years about what needs to be done to advance research in our somewhat small field. These ideas may very well be applicable to science in general, but I'll focus on the relevance to the subject area. I'll start with a couple of notes that reflect the technical side of things and follow with some general thoughts on the state of our research practices.
Vehicle Model Validation and Creation
I'm not convinced that we have realistic enough models of the vehicle (which may or may not include the passive dynamics of the rider). This is especially true of the bicycle and probably even true of the motorcycle. The bicycle data I've collected clearly shows that the Whipple model is deficient in modeling the input steer torque to response relationships. On the other hand, the motorcycle folks have significantly more data to back up their models, but we still have relatively little data to have strong proof of the validity of these open loop models. For the Whipple model, the most likely culprit is our poor assumptions about the tire/ground interaction but the effect of the rider's biomechanics also plays a large role in the bicycle's response. We need more data to prove which models are good predictors.
Engineering is infamous for just showing the best plot in papers. I'm not sure why we collectively accept absolutely no statistical significance for our published results time after time. Our sub-discipline is highly at fault for this. The number of papers I've read with no stats ashames me (including mine!). When are we going to catch up to the rest of science? The data driven approach with proper statistical analyses is the new norm in other sciences, engineering should be no different. We have to set a higher bar in our reviews to encourage more data with more proof that our models are realistic predictors. n=1 is no longer acceptable.
Understanding the Human
Humans are the ultimately complex machine and their role in typical single track vehicles is great. Strides in understanding the human will start to unlock some of the important questions we are all after including the link to the system's properties and the vehicle handling. We've recently published a paper in IEEE Systems, Man, and Cybernetics that touched a bit on how we may assess task independent handling qualities of different vehicles analytically. But this method requires a realistic model of the human's control system and response to error. Better characterization of the human is the only way to understand what handling qualities are.
But I'd also like to point out that the view from the control systems perspective is too narrow. The human is a highly variable and adaptive controller, not to mention that it is one of the most complex "machines" we have to study. For example, we are able to quickly tune our control strategies when given different bicycles and motorcycles, but we also perceive changes to vehicles if the correct stimulus is provided even though no relevant changes were made to the vehicle. The complexity of the human as a controller spans many fields:
- Psychology (cognitive, psychophysics)
- System Identification
- Control Systems
- Human Factors
We need to build connections and collaborations among these fields to make solid headway in understanding human response while operating a bicycle or motorcycle. These collaborations with an increase in experiments and data collected human response under controlled and uncontrolled conditions for a variety of typical maneuvers is desperately needed to start to create and validate realistic human control models.
Now, I'd like to touch on some topics that are very interesting to me. I've had my finger on the pulse of open science for some years now. I'd like to see the bicycle/motorcycle dynamics community to embrace open science practices to encourage faster development and reproducible, reusable research.
We need to share data and we need to produce a lot of it. Lets turn the single track vehicle problem into a big data problem. This means putting highly detailed and documented data into public repositories and embracing the emerging idea of Data Papers (i.e. writing papers strictly about high quality data along with publishing the data so it is easily reusable). Sharing our data and developing some standards in data formats and types would allow us to more rapidly generate models, validate models, and verify the that our models can predict a large amount of data from a variety of motorcycles and bicycles.
We all measure the dynamics of vehicles and the physiological data of human riders. I'm envisioning a common data repository with these measurements collected, documented, and detailed for easy querying and retrieval. I'd like to have the ability to dream up a new model and validate it against hundreds of data sets one night while I'm sleeping. Or use hundreds of data sets to generate or identify a model with machine learning techniques or system identification. As it stands now, we all collect our own tiny set of data and use it with our models. But as a whole we hardly know if our models predict other datasets. Many fields such as genetics, ecology, astronomy, climate change, etc have embraced big data and their predictive ability reflects it.
Let's create a repository that is widely usable.We data from more fully instrumented (kinematic and kinetic) vehicles with human isolation and human in the loop from a variety of vehicles to provide rich datasets. Tire data is also a big missing piece of the puzzle. Psychological data too.
The more we can fill standardized, publicly accessible databases full of rich, quality data, the ability to synthesize and create data driven models becomes real. I believe we need to stop tyring to make your model fit the data and start letting the data create the model. The system identification and machine learning communities are at the forefront of this. Sensors are cheap, data is cheap let's flood our space with good data that is reusable by others.
If we open up our data for the sake of advancing our collective research, what about the software. Why do we all write our own hidden proprietary code? Are we ashamed to show the internals? How confident are we that it is bug free and is correct? Software should be just as open as the equations in our papers. We need to know that the software we are using is producing correct results and we need to have the ability to hack the code. Secondly, why do we all write our own software? Why can't we work together on a large meta software package for bicycle/motorcycle dynamics that has a variety of models, benchmarks, control system implementations, and have it linked to an open repository of data? The collective work of our small but talented group could make a software project that puts FastBike, VehicleSim, and JBike6 to shame. Some of the most innovative disciplines are doing this. The machine learning community is a prime example, so is astronomy and many genetics projects. It makes no sense for us all to continue to write our own versions of the same code everyone else is writing when code reuse is to our advantage for reproducibly, productivity, validity, and minimizing bugs and errors. The computer science world has made all of this relatively easy. The tools are there, we only need the will.
The bicycle and motorcycle literature is actually still manageable. The entire number of bicycle/motorcycle specific papers is somewhere between 500 and 1000 papers. The number of papers has been growing significantly in the past several years. Why don't we have a central repository that is collectively maintained? There are numerous modern software solutions that allow this kind of thing. The time has passed when we all maintain our own reference databases, this is something that is ideal for crowd sourcing and as the papers increase in number it will be the only option.
Very few of the experiments in our field are explicitly reproduced and I'd guess that many of them can't be reproduced. It is well known now that much of science is not actually reproducible. I'd like to see some of the important works from our literature base be reproduced with modern experimental techniques to see if the results hold water.
With our literature base growing closer to 1000 papers we need to spend some time working on more meta papers. Deep analysis and review of the collections of work on vehicle identification, control design, etc need to put into a holistic context so that we can start to believe in the validity of various models.