Linlithgow had lost their first two league matches but, although Penicuik started brightly, the visitors kept possession better and took the lead in the 11th minute.
A cross from the left wing was headed back across goal by Thom and Kelbie slammed the ball home from the edge of the six-yard box.
In the 20th minute, a Kelbie free kick was well saved by Bejaoui. At the other end, McCallum sent over a dangerous cross, but keeper Barnard nipped the ball off the head of Lough.
In the 25th minute, a great run down the right by Forbes saw him get in behind the Rose defence, but his cross along the six-yard box found no takers.
Eight minutes later, Barnard dived at the feet of McCallum to deflect away his shot, and Scott, following up, saw his strike blocked by Donaldson’s outstretched boot.
Then a great run and pass by centre-half Young released McCallum. He rounded the keeper despite being clipped by him, only to see his chip on goal hit the crossbar and cleared.
Minutes into the second half, Shirra dispossessed Barr on the halfway line. He drove forward, set up Coyne, but his shot was tipped over the bar by Bejaoui.
In the 49th minute, Penicuik had a great chance to equalise, but Somerville just failed to get on to the end of a Lough cross.
Linlithgow doubled their lead in the 64th minute when Kelbie again did the damage, heading home a free kick. Two minutes later, Kelbie should have completed his hat-trick but blasted the ball high over the bar when left completely unmarked.
In the 69th minute, a long clearance by Bejaoui set McCallum free, but his powerful left-foot shot was turned round the post by Barnard. From the corner, Forbes headed wide of the upright.
Then, a Hume cross picked out McCallum, but his header sailed high over the bar.
Linlithgow substitute Weir should have added a third, but his weak shot was easily saved by Bejaoui.
In their next attack, Coyne beat Bejaoui, but the ball hit the bar and went over for a goal kick. One minute before full-time, Coyne again got the better of Bejaoui with a lovely chip but the ball hit the underside of the crossbar and came out.
In their last attack, Kelbie again had a great chance to complete his hat trick, but with only keeper Bejaoui to beat he snatched at his shot and it went harmlessly wide.