Hi ElishaIwendi,

Kudos to your efforts of annotating 600 images. I believe those are either satellite or Aerial images. Also, U-Net is a semantic segmentation model. If you want to use it to classify crops you will first have to segment it from the image. The segmentation model will give you the crops of the crop (precise locations of crops in the image) and then you can use a lightweight object classification model to classify the generated crops by U-Net.

I would suggest you take a simple approach to using the Detectron-2 Point Rend model that has instance segmentation. If you take up instance segmentation there will be no need for a classification model given your recall and precision are good. Your F1-Score should be good as you are training for just one class but please watch out for the background and foreground class imbalances.

All the best.