We study the decays B → K ( * ) ℓ + ℓ − and B s → ℓ + ℓ − (ℓ = e, µ) in the minimal supersymmetric standard model with R-parity violation (RPV). From the recent measurements of their branching ratios, we have derived new upper bounds on the relevant RPV coupling products, which are stronger than the existing ones. Using the constrained parameter space, we predict the RPV effects on the forward-backward asymmetries A F B (B → K ( * ) ℓ + ℓ − ) and the branching ratios B(B s → ℓ + ℓ − ). Our results of the forward-backward asymmetries agree with the recent experiment data. It is also found that B(B s → ℓ + ℓ − ) could be enhanced several orders by the RPV sneutrino exchange.The RPV effects on the dilepton invariant mass spectra of B → K ( * ) ℓ + ℓ − and the normalized A F B (B → K ( * ) ℓ + ℓ − ) are studied in detail. Our results could be used to probe RPV effects and will correlate with searches for direct RPV signals at LHC.