Pentecost Sunday is a remembrance and celebration of when God poured out His Holy Spirit upon the early church. When He did this, he was fulfilling what John the Baptist foretold when he said there would come later one who baptized with the Holy Spirit and with fire (Matthew 3:11).
This was also a fulfillment of the promise Jesus made to His disciples regarding Him
sending a comforter, the Holy Spirit (John 14:26).
But neither of those are the reason it is called Pentecost. For this reason we need to look back at the foundation of the Christian belief, Judaism. Jesus and the disciples were Jewish. They observed and celebrated the Jewish Feasts. There are seven feasts celebrated, but three main feasts. The three main feasts are: Pesach (Passover), Shavuot (Weeks or Pentecost), and Sukkot (Tabernacles, Tents, or Booths).
Celebrating the Feast of Pentecost is why the Apostles and other disciples of Jesus were gathered together in the upper room. The word Pentecost comes from Greek Πεντηκοστή (Pentēkostē) meaning “fiftieth” (50th). It is called this because this feast takes place 50 days following Pesach (Passover).
Biblically speaking this feast is tied to the grain harvest, specifically wheat. This harvest lasted seven weeks and was considered a time of joy (see Exodus 34:22, Deut. 16: 9-11, Isa. 9: 2 Jer. 5:24). It was the first day that the people could bring the first fruits as an offering to the Lord.
Later in time it would become associated with the giving of the Torah, or Law, to Israel at 
Mt. Sinai, following their salvation from Egypt. This is largely because of its association with Pesach (Passover) and the obvious ties to the Exodus through Moses and the subsequent giving of the Torah to the Israelites.
In the New Testament, Pentecost is looked at as the birth of the church. It was this event that Jesus referred to when He told the disciples, “Do not leave Jerusalem, but wait for the gift my Father promised, which you have heard me speak about. For John baptized with water, but in a few days you will be baptized with the Holy Spirit” (Acts 1:4, HCSB).
The 1st chapter of Acts tells us there were 120 gathered together. Luke then describes in 
chapter two the Holy Spirit falling upon them with what looked like tongues of fire. It says they were all baptized with the Holy Spirit and spoke in other languages or tongues. Peter later testifies in front of the crowd that this gift of the Holy Spirit is the fulfillment of Joel 2:38-32.
“this is what was spoken through the prophet Joel: And it will be in the last days, says God, that I will pour out My Spirit on all humanity; then your sons and your daughters will prophesy, your young men will see visions, and your old men will dream dreams. I will even pour out My Spirit on My male and female slaves in those days, and they will prophesy. I will display wonders in the heaven above and signs on the earth below: blood and fire and a cloud of smoke. The sun will be turned to darkness and the moon to blood before the great and remarkable Day of the Lord comes. Then everyone who calls on the name of the Lord will be saved” (Acts 2:16-21)
The gift of the Holy Spirit is the catalyst that changed the disciples forever. The truth of the resurrection was burned into their hearts and the group of men who scattered during the arrest and crucifixion, the men who denied with cussing that they even knew Jesus, went on to each give their lives, literally, to see the world turned upside down. Check out the above link for how each disciple gave their life as well as other Christians.
This was the birth of the church, we were to grow and prosper since that time. In many ways we have, in others we have grown stagnant. We need a fresh infilling of the Holy Spirit in order to truly live out the final instructions of Jesus to His church. It is a because of having access to the Holy Spirit that we will be able to do the things Jesus did, and even more, as according to what Jesus said in John 14.
This is why we celebrate the gift that is the Holy Spirit on Pentecost Sunday. May it be more than a side comment or even a potluck dinner to commemorate it. Let it be a push to seek for more of the Holy Spirit and fire operating in our life so that we can make a difference in the world.
A Rock Between The Eyes 