A novel scalable and privacy-preserving distributed parallel optimization that allows the participation of large-scale aggregation of prosumers with residential PV-battery systems in the market for the ancillary service (ASM) is proposed in this paper. To consider both reserve capacity and reserve energy, day-ahead and real-time stages in the ASM are considered. A method, based on hybrid Variable Neighbourhood Search (VNS) and distributed parallel optimization is designed for the day ahead and real-time optimization. Different distributed optimization methods are compared and designed, and a new distributed optimization method based on Linear Programming (LP) is designed that overcomes previous methods based on integer and Quadratic programming (QP). The proposed LP-based optimization can be easily coded up and implemented on microcontrollers and connected to a designed Internet of Things (IoT) based architecture. Both day-ahead and real-time proposed optimization methods, by allocating the computational effort among
local resources, are highly scalable and fulfil the privacy of prosumers.