Fighters from the Islamic State West Africa Province (ISWAP) attacked the convoy with rocket-propelled grenades on Sunday in the town of Mainok outside the regional capital Maiduguri before storming the nearby base, the two sources said.
The attack was the deadliest this year against Nigeria's army which has been battling a decade-long jihadist insurgency in the region that has killed 36,000 people and displaced around two million from their homes.
"We lost 31 soldiers, including their commander who was a lieutenant colonel, in the ambush by the terrorists," a military officer said about the attack which happened around 1100 GMT.
Read more: Jihadists kill dozens in Nigeria attacks: local sources
The convoy was transporting weapons to Maiduguri when it came under attack, said a second military source who gave a similar toll.
"The terrorists came in several trucks, including four MRAPs (Mine Resistant Ambush Protected vehicles) and engaged the convoy in a fierce battle," the second source said.
The militants overwhelmed soldiers, leading to the "colossal loss" of troops, he said.
"We lost a lot of men in very gruesome way."
The jihadists seized weapons and two MRAPs in the attack before overrunning and partially burning the base outside the town, the two sources said.
Mainok, about 50 kilometres (30 miles) from Maiduguri, has been repeatedly targeted by the jihadists.
ISWAP has frequently set up bogus checkpoints along the 120-km highway linking Maiduguri and Damaturu in neighbouring Yobe state on which Mainok lies, killing and abducting travellers.
ISWAP split from mainstream Boko Haram faction in 2016 and rose to become a dominant force in the northeast as Nigeria's conflict has spilt over borders into neighbouring Chad, Niger and Cameroon.
Targeted killing
Around 2,000 residents of Geidam in neighbouring Yobe state fled their homes after a separate attack by ISWAP who invaded the town Friday, local officials said.
On Sunday residents fled, crossing to the other side of the river to escape militants who have been in control since they attacked the town.
"Our people are fleeing Geidam which is now under the control of the insurgents," Ali Koko Kachalla, Geidam's political administrator, said.
Late on Friday ISWAP jihadists took control of Geidam after a gunfight with troops, looting and burning stores in the town.
At least 11 civilians were killed in the fight after a projectile hit two adjoining homes, killing all occupants, according to residents.
The exodus was prompted by targeted killings of residents by the insurgents, according to fleeing residents.
"Everybody is running away because the insurgents have started killing people who are Christians and those with western education," resident Babagana Kyari said.
"So far they have killed two Christians and two Muslim school teachers. They went to their homes and slaughtered them," said another resident Ari Sanda.
The army issued a statement on Saturday, claiming to have taken back Geidam from the jihadists. But residents and local officials said militants were still around.
Since 2019, soldiers have closed up some smaller bases and moved into larger fortified garrisons known as "super camps" in an attempt to better resist militant attacks.
But critics say the "super camp" strategy has also allowed militants more freedom in rural areas and left travellers more vulnerable to kidnapping.
Follow us on Facebook, Twitter and Instagram to stay connected