The Early Church did not question Mary's perpetual virginity. They believed that Jesus was miraculously conceived by the power of the Holy Spirit; that she was a virgin before, during, and after the birth of her Son, Jesus and remained a virgin.
They knew that Scripture referred to Jesus as the son of Mary and not a son of Mary in Mark 6:3. Neither Hebrew nor Aramaic had a word for cousin. Our colleagues at Catholic Answers tell us:
Because neither Hebrew nor Aramaic (the language spoken by Christ and his disciples) had a special word meaning "cousin," speakers of those languages could use either the word for "brother" or a circumlocution, such as "the son of my uncle." But circumlocutions are clumsy, so the Jews often used "brother." (Read more)
In John 19, Jesus entrusted the care of His mother to the Apostle John at the foot of the cross (John 19:26-27). If He had siblings, that responsibility would have been transferred to one of them.
The Church's Scriptures that support Mary's Perpetual Virginity:
The Old Testament prophecy fullfil in these passages: Behold, a virgin shall conceive and bear a son, and his name shall be called Emmanuel (Isaiah 7:14)
18 Now the birth of Jesus Christ took place in this way. When his mother Mary had been betrothed to Joseph, before they came together she was found to be with child of the Holy Spirit; 19 and her husband Joseph, being a just man and unwilling to put her to shame, resolved to send her away quietly. 20 But as he considered this, behold, an angel of the Lord appeared to him in a dream, saying, "Joseph, son of David, do not fear to take Mary your wife, for that which is conceived in her is of the Holy Spirit; 21 she will bear a son, and you shall call his name Jesus, for he will save his people from their sins." 22 All this took place to fulfil what the Lord had spoken by the prophet: 23 "Behold, a virgin shall conceive and bear a son, and his name shall be called Emmanuel" (which means, God with us). 24 When Joseph woke from sleep, he did as the angel of the Lord commanded him; he took his wife, 25 but knew her not until she had borne a son; and he called his name Jesus.
Matthew 1 18-25
The inspired Scriptures tell us Jesus was the son of Mary, not a son of Mary
3 Is not this the carpenter, the son of Mary and brother of James and Joses and Judas and Simon, and are not his sisters here with us?"
Mark 6:3
The Angel Gabriel explains to Mary how she will remain a virgin and still be the mother of Jesus
26 In the sixth month the angel Gabriel was sent from God to a city of Galilee named Nazareth, 27 to a virgin betrothed to a man whose name was Joseph, of the house of David; and the virgin's name was Mary. 28 And he came to her and said, "Hail, full of grace, the Lord is with you!" 29 But she was greatly troubled at the saying, and considered in her mind what sort of greeting this might be. 30 And the angel said to her, "Do not be afraid, Mary, for you have found favor with God. 31 And behold, you will conceive in your womb and bear a son, and you shall call his name Jesus. 32 He will be great, and will be called the Son of the Most High; and the Lord God will give to him the throne of his father David, 33 and he will reign over the house of Jacob for ever; and of his kingdom there will be no end." 34 And Mary said to the angel, "How shall this be, since I have no husband?" 35 And the angel said to her, "The Holy Spirit will come upon you, and the power of the Most High will overshadow you; therefore the child to be born will be called holy, the Son of God. 36 And behold, your kinswoman Elizabeth in her old age has also conceived a son; and this is the sixth month with her who was called barren. 37 For with God nothing will be impossible." 38 And Mary said, "Behold, I am the handmaid of the Lord; let it be to me according to your word." And the angel departed from her.
Luke 1:26-38.
Mary is entrusted into the Apostle John's care, which would never happen if Jesus had other brothers or sisters (something the Scriptures make no mention of. )
26 When Jesus saw his mother, and the disciple whom he loved standing near, he said to his mother, "Woman, behold, your son!" 27 Then he said to the disciple, "Behold, your mother!" And from that hour the disciple took her to his own home.
John 19:26-27