It's one of two things: the plaster wasn't cured enough or you didn't prime enough. And now you have another problem: you PVA-ed your walls.
Since you plastered over brick, a bonding compound (like PVA) was likely to have been used to help the plaster adhere to the brick (hopefully, or you'll have a whole other set of flaking off problems down the road). A bonding compound lengths the normal drying time of plaster by a couple of months. I know it seems like forever, but keep in mind that people used to be told not to paint for a year after plastering to allow for a full cure (for a 3-coat job on wood lathe, before central HVAC).
Paint flakes off of plaster because the plaster absorbs the water in the paint so fast that the paint doesn't get "roots" on the plaster. Hence, you prime with diluted emulsion paint (% varies according to the paint); usually this takes several coats to properly "seal" the plaster so it will take topcoat paint. Generally speaking, you don't need to "seal"/prime plaster other than with a diluted emulsion paint. Sealing with PVA is NOT a good idea if you want to paint over it... and you didn't say, but I really really hope it was diluted. Yes, diluted PVA is used between coats of plaster, but paint will strip right off of it. A few people have had no problems using some really diluted PVA as a sealer, but they were lucky and probably will have problems later anyway. Using PVA compound proper (undiluted) is um, bad at any point.
Anyway, here's your situation: the plaster was not cured enough or it was not primed enough (this can take up to 6 coats btw, depending on your paint and plaster -- bonding/browning coats will require more priming). Either case will cause paint to flake off. Now you have PVAed your walls and, naturally, the paint is coming off. What you should do: sand your walls as much back to plaster as you can and paint on a microporous paint (after cleaning the walls thoroughly). Microporous paint will allow the plaster to breathe, but still give you a finished look to the walls. If it doesn't stick, sand some more and try again. If it sticks, live with it for the next several months, then try priming (enough) and painting again. Since you PVAed your walls and it's not for sure that it was cured, you should wait until late summer to finish out painting the walls (with several coats of diluted emulsion paint) and then topcoat. The PVA will slow the drying process and it's unlikely that you can remove ALL of it (since it sinks in) by sanding, so several months will be necessary. Of course, there IS the option of removing the plaster altogether and redoing the entire job.
Note: use matte emulsion topcoats when you can. If you want gloss of any amount, wait 6 months-year to be sure the plaster is fully cured first and then use a vinyl emulsion (not oil; usually goes well but sometimes has problems). Even if the paint SAYS it can be used over new plaster, remember that "new" is a relative term and that it's better to live with a matte coat for several months and do it right.
BTW: you primed with a "Durabond solution"?? Do you mean you tried to use a diluted drywall compound or plaster product as a primer??? Great for getting smooth walls, but you'll still have to actually prime the walls --it doesn't prime itself... and no primer = flaking paint.