Train pre-trained yoloV8 model for more epochs

[Peter-coding]

Dear Ultralytics,

I have finished a 100 epoch training on a yolov8s model, on my custom dataset, with:

yolo task=detect mode=train epochs=100 data=data_custom.yaml model=yolov8s.pt imgsz=640 batch=8 workers=8 device=0

After finishing, I got good results:

I want to train the finished model for more epochs, I want to extend it by 100 more epochs.

I run a new training on the finished model, with last.pt (on the same custom dataset, I changed nothing):
yolo task=detect mode=train data=data_custom.yaml model=last.pt epochs=100 imgsz=640 batch=16 device=0 workers=0 patience=30

Results of the second training is:

As you can see, the train/box_loss, train/cls_loss and train/dfl_loss increased after the second training, but the others increased:

epoch	train/box_loss	train/cls_loss	train/dfl_loss	metrics/precision(B)	metrics/recall(B)	metrics/mAP50(B)	metrics/mAP50-95(B)	val/box_loss	val/cls_loss	val/dfl_loss	lr/pg0	lr/pg1	lr/pg2
100	0.14602	0.14118	0.80761	0.92354	0.92618	0.94079	0.92918	0.14608	0.41871	0.49317	0.000199	0.000199	0.000199
185	0.1757	0.2764	0.83431	0.93837	0.92103	0.9453	0.93317	0.15178	0.38881	0.51036	0.001684	0.001684	0.001684

The learning rate on training 1 started from ~0,00333146 and went to 0.000199 on epoch 100.
The learning rate on training 1 started from ~0,00332959 and went to 0,001684 on epoch 85 (I set patience to 30, thats why it stopped on epoch 85 of the second training)

My questions are:

• Is this the right way to train for more epochs on a finished training model? I read that I don't need to use resume=True if the model is finished the previous training. Is it true?
• I didn't change the learning rate (I didn't touched it) for the second training. Is it a problem for the second training?
• I made a diagram from the first and second training. Does it make sense to put together 2 trainings on the same diagram? Does it have correlation?

Second training start:

I have saved a copy of the first training's best.pt and last.pt. Do you think I need to train it from there again, with an other view of point? What should I adjust?

Thank you,
Peter

[UltralyticsAssistant]

👋 Hello @Peter-coding, thank you for reaching out and for using Ultralytics 🚀! We appreciate your detailed message.

For extending your training, your approach seems logical, but there may be some nuances 🧩 to consider. An automated response like this can't fully address them, but fear not, an Ultralytics engineer will be on hand soon to provide more specific guidance.

If this is a 🐛 Bug Report, we recommend sharing a minimum reproducible example if there are concerns about unexpected behavior during your training phases.

For optimizing your training process, please ensure that your environment is up to date. Upgrade to the latest ultralytics package along with all required dependencies in a Python>=3.8 setup with PyTorch>=1.8:

pip install -U ultralytics

Take advantage of the power of our community to further enhance your learning and model-building journey. Join us on Discord 🎧 for conversation in real-time, or discuss your in-depth technical inquiries on Discourse.

Lastly, if you're eager to experiment in different environments, feel free to explore any of these up-to-date setups which come preconfigured:

• Notebooks with free GPU:

We hope this helps, and we're excited to see what you accomplish with another round of training! Stay tuned for more personalized insights from our support team. 🌟

[glenn-jocher]

@Peter-coding hi Peter,

To continue training, using last.pt without resume=True is correct. The learning rate should be adjusted if performance degrades; consider reducing it for stability. Combining training results in one diagram is fine for comparison. If issues persist, you might try starting from best.pt and adjusting hyperparameters like learning rate or batch size. For more details, visit Ultralytics Training Documentation.

[Peter-coding]

I used batch=8. do I need 16?

These are the settings:

Parameter	Value
lr0	0.01
lrf	0.01
momentum	0.937
weight_decay	0.0005
warmup_epochs	3.0
warmup_momentum	0.8
warmup_bias_lr	0.1

With what amount should I decrease these for new training on the first trained model? Do I need to use pretrained parameter?

Thank you,
Peter

[glenn-jocher]

@Peter-coding hi Peter, batch size depends on your hardware capacity; 16 might improve stability if your GPU can handle it. For retraining, consider lowering lr0 slightly for stability. Using pretrained=True can help if starting from scratch.

[Y-T-G]

You can follow this

https://www.reddit.com/r/Ultralytics/comments/1eolwl8/the_correct_way_to_train_from_a_previously/