Debriefs

2023: Bordeaux, France

What went wrong

Hardware

• Heads fell off
• Servo blowing when tuning
• Overload turn on not off!
• Printed knee mounts PLA breaking, onyx good tho
• OpenCR died, issues with OpenCR
• Need more battery extension cables, one broke
• NUC might be dead? Unknown, maybe cable or power. Not OpenCR because Kevin is fine with the OpenCR from it.
• Ethernet port breaking, ethernet cables being annoying/breaking
• Cables came out a lot and broke a lot
• Damage a lot when falling, parts snapping, such as head - maybe falling relax would help
• Voltage in HardwareIO, other HardwareIO bugs
• Redesign back, button panel is annoying, the cables on a removable part is annoying
• Too many cables in the torso!
• Decide on foot studs - new base? Rubber underside/shoe
• Red button on some robots inconsistent/didn’t work, and other buttons
• Bolts coming loose a lot! Neck and hips.
• Robots are a little different
• Knee spring mounts breaking easily
• Can’t connect two cameras at once because they are on the same bus
• Power connection hard to get to in robot when back plate on

Motions

• Walk didn’t handle the bumpy ground well, ground was uneven in Bordeaux
• Falling relax not on
• Couldn’t get up
• Couldn’t kick

Networking

• Beginning of comp needed to iron out some issues with understanding how to connect to fields - drop in games are difficult for networking!!
• NUsight could be used to a bigger potential
• NUsight unreliable in the lab
• People not working on NUsight

Vision

• White balance, annoying to put the white paper in front
• Visual mesh failed to train with harder dataset, no robot detection

Localisation

• Bug in second half where it thinks its on the wrong side of the field

Team Communication / Behaviour

• Don’t want to collide so better behaviour for that and robot detection
• Walk path planning kinda dodgy and align to ball not great
• Diving all over the place

Organisational

• Didn’t have enough time to test and tune because we implemented many new things - new legs, new odometry, new behaviour system, new nucs, new subcontrollers, new localisation
• User experience for running games - buttons not working and could be better used for running games, eg localisation reset
• Duplicate code and build times high
• Some people not well trained in how to code in the main codebase
• Get air conditioned rooms next time

What went well

WE WON A GAME!

Hardware

• X Series knees performed well when walking and standing
• New hips held up well
• Can get up from front or back now without rolling over
• Hips hot swappable
• Computational power good, no issues
• Cooling on nucs good, didn’t cook any
• We had a subcontroller to use on all robots
• Despite bugs, OpenCR did work generally

Motions

• Walk was acceptable, nice wiggle

Networking

• GameController connection flawless
• Wifi worked, networking good, way better than last time

Vision

• Worked without any retuning
• Field and field lines were really good
• Balls were detectable after some tuning

Localisation and Odometry

• Odometry worked
• Localisation worked most of the time

Behaviour

• Was good, worked with basic walk ball to goal
• Ready was good with localisation
• No Director seg faults, worked well

Organisational

• ROBOTS ARRIVED!!
• AIR TAGS WORKED!!
• Tape used well
• Game kit good
• Team work, coming to games
• Trent got on TC
• Jason liked ref-ing
• People knew what to do for the most part

What’s Next

• Stagger NUSense in robots rather than switch them at the same time
• Maybe use localisation to determine when to throw out balls
• NUsense, no OpenCR
• Kicking
• X-Series in the ankles
• Ankles shock absorber springs, ankle forward so force is on centre of foot
• Foot out of aluminium
• Feedback based control, balancing, for kicking and walking
• Redesign hips, weight reduction, baring design
• Torso design, buttons, space, extension on nuc ports so damage is on the extension not the nuc, maybe include in the extension routing for the usb to get another camera bus
• Vision for goals and robots
• Directional kicks later
• Safety measures eg buzzer, rotational limits
• Culling codebase
• Aluminium head mounts
• Analysis and research on design of hardware
• Tom wants faster development
• Build times, more dev ops
• Falling relax
• Bug fixing and ironing out things
• Heatsinks for the knee servos

2022: Bangkok, Thailand

RoboCup Bangkok was the first in-person competition in three years due to COVID-19. All but one team member had never been to an in-person RoboCup competition, and none had been to an overseas RoboCup competition. The team faced further issues when the majority of the luggage did not arrive until the end of the first competition day.

What went wrong

Hardware

• Didn't pack enough cables for battery chargers
• CM740s are no good, need to get something new. Cannot buy more, and we only have two working.
• NUC died due to overheating
• Needed more monitors: one was just not enough
• Issues of wiring getting in the way – makes it hard to swap things out
• No soldering happened, but the setup could have been better if we did use it
• Long ethernet cables became tedious to use and untangle
• Heat dissipation in the torso is an issue

Motions

• Rollover from back to front, then using get up front does require a lot of floor space and caused a few tangles, such as the robot getting its feet caught in the goal net.

Networking

• One robot had broken WiFi antenna, preventing it from connecting to GameController
• Interference in the venue made it sometimes hard to connect to GameController

Vision

• Cameras had a red tinge due to automatic white balancing.
• New mesh is too computationally expensive for the NUCs that we have. Potentially go down to one camera per robot and lower the number of images received from the camera
• The bottom of the mesh, near the robot's feet, bends around, possibly causing issues with ball detection near the feet
• Hoc was slightly off - odometry or camera calibration
• NUsight was unreliable for testing vision
• The 360 degrees Theta camera used for collecting vision data was in check-in, not carry-on, so the team couldn’t collect data until later. The team did get one image earlier using a Theta camera from CIT Brains

NUsight

• Difficulty connecting to NUsight

Localisation and Odometry

• No localisation
• Hoc was slightly off for vision

Miscellanious

• No robots until end of first competition day due to missing luggage

What went well

Hardware

• Didn’t replace a single servo at the competition
• Padding was effective for keeping the robot protected and for colour requirement in inspection

Motions

• The robots were able to walk stably!!
• Getups basically worked flawlessly

Networking

• Able to connect to GameController with at least one robot for most of the games

Odometry

• Pitch and roll were good, better than the previous year

Vision

• Fine tuning for vision worked well despite not having our own Theta camera

Teamwork and Moral

• Got robot ready for qualification very quickly
• We all came to the games - had three different robot handlers for four games
• Got lots of promotional footage of the robot
• We made the quarter finals despite not having two whole set up days and the first competition day to test the system and networking
• Within 30 minutes of getting our robots from the airport, the robots were walking and getting up
• Good preparation before the competition was vital to the team being able to perform at all when the robots arrived at the end of the first competition day
• Overall made good progress compared to RoboCup 2019 Sydney
• On par or better than Team-KMUTT, UTRA, Hamburg Bit-Bots, Bold Hearts, ITAndroids, RoboFEI
• Students at home were around to help with vision fine-tuning

What’s Next

Networking

• System configuration tool
• Configurable headless mode, lightweight, if no system config tool due to it being tedious to set up networking
• Team Robot communication – only one robot connects to GameController and then sends information to the other robots
• Make GameController connection more robust
• See how the small robots do team communication

Hardware

• Find a replacement for CM740’s quick so we can buy or make them before the end of the year.
  • Possibly redesign NUsense to only contain the core functionality. Additional functionality can be added with daughterboards/shields, creating a more modular system.
• Get a decent battery powered soldering iron (TS100), a decent stand, sponge, and safety glasses with some back up wire and various connections. Ryobi may have some interesting products.
• Better servos on the knees
• Kneecaps to stop the knees overextending
• Servo covers - couldn't get up from the back because of servos hitting each other
• Improve cable management in the robot
• Sort out a reel for longer ethernet cables
• Look into purchasing small portable monitors for RoboCup
• Look into more off the shelf solutions for hardware, such as a proper cooling system that we can buy off the shelf
• Redesigning the back of the robot for easy access and able to give better padding protection

Organisation

• Target research / postgrad students - remove ‘maybe’ from do you plan on doing postgrad question
• Attempt the git workshop again
• Project management. Make a physical board
• Create a culture of high performance
• Keep momentum up, particularly post Robocup
• Try and keep dev evenings early in the week (Monday, Tuesday) so people aren't busy with assignments
• Make team desirable for academics
• Come up with ideas on how to improve motivation
• Keep dev evenings / time in the lab focused on development
• Pack robots whole rather than disassembled - 2 in large case, one in square case, possibly carry-on

NUsight

• Sort out general connectivity to NUsight for testing systems - fix issue that prevents consistent connection

2021: Online Simulation

Due to COVID-19, the 2021 competition was held online using the simulation software Webots.

What went wrong

Walking

• We underestimated dev time required to get walk optimiser working, so put dev resources on that that might have been better spent elsewhere
• Better path planning is needed as the robot just walked straight towards the ball (now that we have strafing and pivoting abilities we should incorporate this)
• An closed loop walk needs to be implemented. The walk can have sensor feedback, but it has not been enabled because odometry was not working (until now)
• The walk command that was sent and the actual output of the walk engine was different

Vision

• The Visual Mesh had been updated since last RoboCup, and we didn’t allocate someone to merge the update into our codebase and test it until after vision data was collected - this should have been done in parallel
• Vision data collection ran later than scheduled
• Not enough people know how to run the vision pipeline through to something like NUsight for verification (needs more docs)
• When we went for the ball, we made a complete bee-line for the ball and got penalised for walking into other teams. We need robot detection and avoidance!!

Odometry

• Yaw was not working, not enough members of the team understand how Odometry works
• Still needs tuning

Localisation

• NaN's in BallLocalisation due to an invalid initialisation lead to us to having a NaN walk command at times
• Robot localisation always assumed goal posts were opponent posts because of ancient (pre-2015??) code
• All of localisation was old code that had not been tested in eons due to us being unable to test for many years
• Localisation wasn’t used in our final game because of uncertainty/issues in the system. Mostly from our goal detector and head behaviour systems

Behaviour

• All of our robots would stop when one robot was penalized in game
• Once we get in a stuck state, we don’t have a way to “reset”
• Webots would teleport our robots when they were penalised which messes with our system. Lots of different components required flushing and not all were caught.
• The walk engine was running faster than walk path planning which was running faster than soccer strategy. On a new state change from Game Controller, walk keeps running many times before soccer strategy executes. Only really an issue when the robot was teleported in Webots - related to above, need to keep everything flushed for a small amount of time until all the systems catch up
• Head behaviour just looks for balls and doesn’t change to look for goals. Problematic with the small field of view we had in Webots.
• Possible problem with head behaviour since kinematics doesn’t go to the camera anymore, just the pitch servo.

Data Logging

• We need to be more careful about what messages we log. We lost a lot of, potentially very valuable, data because our logs filled up the simulation instance and was discarded. Should run some tests with Webots to see what sort of data rates we are dealing with so we can more easily choose what messages we want/need/can afford to have
• In an effort to stop our logs getting too big we implemented a max size after which all logging would stop. As part of this we changed the FileLogHandler module to a sync group instead of a mutex and lost valuable time solving the resulting segmentation fault

Webots + GameController

• By running Webots on the local network on multiple devices it was necessary to implement a network traffic filter so as not to interfere with other running simulations. One such filter was accidentally committed to a submitted code image for a game and our robots did nothing all game. There should have been a checklist confirmed by at least two people other than the person doing the submission before the final code was submitted
• Random crashes and connection issues ate up valuable development time
• Need Devops, Code Review, and Deployment during RoboCup. Systems were getting implemented without proper testing and revision which lead to unexpected issues and breaking of other modules
• We had a close call for one of the games where our final submission was completed just a minute or two before the deadline. Early and incremental merging, testing, and submissions helped ensure we had a smooth submission.
• Multiple people need to be aware of the building/submitting process so that we aren't relying on a single person be available/awake at critical moments
• The system that we used to handle messages from the RoboCup Game Controller was out of date and incorrect in places. Need to keep up to date with it.
• There were some errors setting up robot-specific config files due to a misunderstanding of where Webots looked for the config file. webots config folders didn’t work, webots1, webots2, webots3 did.

Tools: NUsight, NUbook

• NUsight: Testing with many docker robots was difficult to filter and find our docker robot. A way to differentiate or filter out other images is required
• NUbook: Lots of little things that we were discovering though the dev process like network traffic filter flags, which repo/branch to pull, are hard to document with the normal process. PRs need to be getting reviewed and merged quickly to save time particularly for necessary pages. The process for building the latest version of Webots with the correct configuration was not merged for nearly a month and people were building directly from the PR preview

Team Morale, Participation and New Members

• It’s hard to keep track of what needs to be done, who’s working on what and what our priorities are when the dev cycle speeds up. Makes finding a job daunting if you don’t feel confident to walk up to a group of people and ask them if they would like some input.

Planning and Organisation

• Need to account for exam period a bit more explicitly
• Didn’t have access to back door for a while which made team movement difficult

What went Well

• We set up a good computer to be the dedicated building/commiting/pushing computer. By keeping it up to date it meant that build times remain low and it is always in a good state to make a new submission
• Walking was pretty well tuned especially by the last game
• Ball tracking/path following obtained a passable fix for the games

Vision

• Visual mesh was well trained and we had a working particle filter
• Could use vision data to find and walk towards the ball
• Head behaviour was sorted out to stop the robot from fixating on missing objects
• Goal detector had issues and required a significant amount of vision post-processing
  • Implemented field length distance for goal throw outs
  • Implemented clustering for broken up goals
• Ball detector was working well, false negatives were uncommon

Odometry

• Fixed exploding filter
• Pitch and roll very good

Localisation

• Fixed cholesky decomposition which was causing localisation to stop working
• Ball detector NaN's fixed
• Robot particle localisation improved and we handle goal post measurements better
• We have a new particle filter (hopefully better implemented and with more documentation)
• Field line detection for improved field/robot is mostly implemented

Behaviour

• Identified potential
• Penalty shootout behaviour was really good even though we didn't end up using it in a game
• Passed penalty shootout tests
• Fixed issue in head behaviour that caused the head to stay fixated on where it was when it last saw the ball if it lost sight of it

Webots + GameController

• We now have a working, complete simulator that will be invaluable in future testing and development

DevOps and Code Review

• Incremental submissions to prevent a final rush that may introduce improperly tested solutions

Tools: NUsight, NUbook

• Identified new concepts to include within NUbook based on junior team member code reviews

Team Morale, Participation and New Members

• Had impactful work done by newer members

Planning and Organisation

• Good idea to book a classroom alongside using the lab, was good for watching matches

What's next

• Having a way to rapidly prototype ideas in NUClear (i.e. python modules)
• A “digital whiteboard”
  • Keep track of what needs to be done, and who’s doing what
  • Keep track of specific dev things which don’t fit in NUbook - e.g. when is the next game, everyone needs to ./b target generic again because something broke, etc.
  • Develop a system to assign tasks to people GitHub Issues/Projects? Trello?
• Get newer people confident before RoboCup because when we’re needing to be speedy there isn’t a lot of time to think about helping shy/less confident members.
• Improved walk engine
• Vision: how useful is improving the vision data collection tool? If we use Webots again, maybe? Not useful for real world. Ideas for real world are
  • Multimodal CycleGAN
  • Motion capture data
• All new vision members need to be able to train, run, test visual mesh (with a dummy dataset)
• Consider more vision post-processing including work on green horizon throw outs.
• Optimise odometry values, maybe for when motion capture is set up
• Develop head behaviour to include searching for goals as well as the ball
• Proper state machine for behaviour to handle all cases - Kick-off in Normal, Ball drop, Other things. Infrastructure to begin with rather than actual implementation of what to do in that stage.
• Implement isolated tests in Webots for modules.
• More NUbook pages. Documentation is important for new members to learn things quickly
• We need the Director, a system that will improve our robots decision making when it comes to what behaviours run at a given time
• Have a look at implementing ROS bridge, a system with a huge number of libraries that we can use if we can interface with it
• Investigate PlotJuggler to be used as potential alternative to NUsight charts
• Numerical optimization for Inverse Kinematics
• Highly accurate system identification so our model best translates from Webots to the real world
• Path planning/tracking (vector fields that include obstacles or something similar)
• Develop a simple program that uses sliders for tuning YAML files for our scripts
• Multi-state walk engine, which starts in a slower state where it is more stable, then once it has started moving gradually increment the speed
• Behaviour visualisation. Either some indication in existing NUsight views of the state of the robot, or a new NUsight behaviour view, with some indication of graphs of previous states and possible future states.
• Potentially a different NUsight view for different providers e.g. a NUsight view for the walk engine behaviour provider
• Fix BallDetector to properly let the system know if it can detect no balls
• Set up different configuration files so we debug and test on one set of config files, and play on separate set of configs - to prevent breaking config changes like the network traffic filter
• b tool or command line args for dealing with network traffic filter or some config stuff like that. NUClear has a commandlineargs trigger which could be worth looking into
• We should make PRs to Webots to do stuff like use a different mirror for their deps
• In general, we lack as a team deep collective understanding of how the whole system works. Continuous work will build our confidence and prevent us from having to assume too much. We should be confused if something doesn’t work because we know how it should work, rather than how we assume it should. This can be improved by continuing to improve our system, writing tests and building out the Director. Our current system overview is probably too broad. More documentation will also help with this.

2020: France, Bordeaux

Due to COVID-19, the 2020 competition in France, Bordeaux was cancelled.

2019: Sydney, Australia

What went wrong

Walking

• We were using a legacy walk engine (open loop with no reactive balance control, designed for smaller robots (Darwins))
• The simple walk path planner was previously not configured for iGus and the planner did not tell the robot to go to the correct position when it could not see the ball

Vision

• Wasn’t properly implemented until the second day of the competition.
• There was no integration testing with vision and the rest of the system.
• Green on the field wasn’t detected properly due to low saturation in the images.
• Seg faults were found due to edge conditions with the goal post detection which caused instability/restarts in the walk
• Visual Mesh
  • Couldn’t detect field lines because we couldn’t detect the field (wrong green color, network didn’t have the right training data).
  • Couldn’t see the goals though because they were thinner than what was in the training data.
  • Visual Mesh packets to NUsight were too big, there was lots of dropped packets.

Odometry

• Since moving to the CM740 the axis of the gyroscope and accelerometer changed. This change resulted in uncovering an error in the Kalman Filter, which was corrected during the competition.
• Foot down detection was not accurate
  • neural network training was inadequate to determine foot-downness. Tuning of the walk was performed after training the network, so the trained models were not accurate. The walk was inadequate for getting good network training data as the robots weren’t lifting their feet off the ground enough to get good data - there were lots of noise in the small steps.

Mechanical/Hardware issues

• Problems with servos overloading and locking, possibly due to motor drive shafts bending, or motors magnetically locking to one set.
• Head fell off due to not being bolted in correctly.
• Studs fell off/got stuck in the turf, possibly due to lack of adequate threading and inappropriate studs for our usage.
• Robot disconnected from power supply when it fell over.

Configuration issues

• Multiple layers of configuration (with multiple platforms, robot-specific configurations, and multiple configuration files for each system) led to confusion and made it difficult to tune the walk, vision camera parameters, etc.
• Changing network configurations for different fields could be made faster/easier - perhaps a script that takes params and updates the network configuration correctly

Scripts

• Needed re-tuning because: different fields (turf), different robots (scripts weren’t tuned before RoboCup for the new robots).

NUsight2

• Chart view rendering was slow (especially when running start/dev)
• Network issues with NUsight when trying to run the robots with a static, direct ethernet connection.

RoboCup competition and games

• Team members didn’t go to see most of the games, was demotivating. Robot handling/computer operation during games should be shared or rostered - every team member should be trained on how to do this, to share the load. This year it was tough having just one/two people do it.
• Wrong prioritization of tasks, could have focused more on immediately important things.
• Need a pre-RoboCup briefing/training for team members
  • Handling robots, during games - putting them on the field, when/where to touch, place remove robots, what to say to refs, etc.
  • How to fix robots correctly.
  • How the configuration system works - how to adjust and tune parameters, NUgus specific vs global config, /etc/network/interfaces, robocup services (systemctl) enable disable start stop, globalconfig.yaml, soccerstrategy.yaml, team ID and player ID.
• Post RoboCup de-briefing with a physical debriefing meeting should be done.
• Should write an entry in the team handbook about what went wrong, what was done well, what to do for next RoboCup.

What went well

• NUsight2 was very useful for debugging the odometry filter, less problems than NUsight 1. Had the NUgus models which was good.
• Code errors in odometry problem were fixed, but it still needs tuning.
• Visual Mesh could successfully find the balls when the green horizon was detected on the field.
• Connected to Game Controller most of the time.
• Getup scripts were good - didn’t work when the robot had on the game vest - it couldn’t turn itself over.
• Penalty kicks were good (won 3:0) - did a good job of tuning them well - although they did some damage to the robots.
• New members participated more, learned a lot.
• Recorded and labelled a lot of data which could be used to fix vision and walk issues next year:
  • Recorded sensors (motor positions), gyroscope and IMU data
  • Recorded vision data with visual mesh classification
  • Started work on creating a HDR module for creating HDR images from the robot’s camera - useful for getting more image data with different exposures to train vision. Module needs to be finished
  • Annotated segmentation maps for images obtained from the robot
  • Recorded 360-degree HDR images of the adult sized field
  • Recorded during robocup/during games
  • Recordings are NBS files saved on Trent’s laptop, to be stored on the NAS.
• Starting writing team handbook and more comprehensive documentation

What’s next

• Better testing for next year.
• Finish up initial documentation/team handbook.
• Finish HDR image creation module.
• Re-do the shoulder and hips to be more rigid - so they don’t flex when loaded. Need more rigid connection between body and hip joints.
• Implement new battery system (power tool batteries).
• Finish NUSense (code and hardware changes).
• Finish up the script tuner in NUsight2 for better tuning of scripts.
• Vision
  • Implement field lines detection
  • Collect more diverse/real training training data (from cameras) to train/tune the vision system. More data with less synthetic features in the dataset.
  • Tune/improve of ball and goal detection
  • Visual Mesh training for visual mesh (needs more training data from real robot, varied saturations, hue, exposure, gain.)
• Get a better walk