Mongoose.aggregate(pipeline) länkar flera samlingar med $unwind, $lookup, $group

Det här var svårt för någon ny till MongoDb s aggregate . Jag delar upp mitt svar i steg för att demonstrera för andra som försöker aggregera en array med hänvisning till flera samlingar.

Steg 1 - $match för att filtrera på insamling

$matchen accepterar samma frågor som db.collection.find({}) och returnerar en rad matchande resultat i fallet nedan väljer jag fyra specifika poster här


{ '$match':
     { _id:
        { '$in':
           [
              ObjectId('5f7bdb3eea134b5a5c976285'),
              ObjectId('5f7bdb3eea134b5a5c976283'),
              ObjectId('5f7bdb3eea134b5a5c976284'),
              ObjectId('5f7bdb3eea134b5a5c976289')
           ]
        }
     }
}

$match Result

[ 
  { _id: ObjectId('5f7be0b37e2bdf5b19e4724d'),
    name: 'CAPTAIN_SAIL',
    classes: [ 'sail' ],
    license: 'WC-1',
    watercraftContexts:
     [ { _id: ObjectId('5f7be0b37e2bdf5b19e47241'),
         watercraftType: 'Sailboat',
         ref: 'sailboats' } ],
    __v: 0 },
  { _id: ObjectId('5f7be0b37e2bdf5b19e4724e'),
    name: 'CAPTAIN_YATCH',
    classes: [ 'yatch' ],
    license: 'WC-2',
    watercraftContexts:
     [ { _id: ObjectId('5f7be0b37e2bdf5b19e47242'),
         watercraftType: 'Yatch',
         ref: 'yatches' } ],
    __v: 0 },
  { _id: ObjectId('5f7be0b37e2bdf5b19e4724f'),
    name: 'CAPTAIN_SHIP',
    classes: [ 'ship' ],
    license: 'WC-3',
    watercraftContexts:
     [ { _id: ObjectId('5f7be0b37e2bdf5b19e47243'),
         watercraftType: 'Ship',
         ref: 'ships' } ],
    __v: 0 },
  { _id: ObjectId('5f7be0b37e2bdf5b19e47253'),
    name: 'CAPTAIN_SAIL_YATCH_SHIP',
    classes: [ 'sail', 'yatch', 'ship' ],
    license: 'WC-7',
    watercraftContexts:
     [ { _id: ObjectId('5f7be0b37e2bdf5b19e4724a'),
         watercraftType: 'Sailboat',
         ref: 'sailboats' },
       { _id: ObjectId('5f7be0b37e2bdf5b19e4724b'),
         watercraftType: 'Yatch',
         ref: 'yatches' },
       { _id: ObjectId('5f7be0b37e2bdf5b19e4724c'),
         watercraftType: 'Ship',
         ref: 'ships' } ],
    __v: 0 }
]

Steg 2 - $slappna av så att vi kan iterera med $loopup

I denna resultatuppsättning finns det en array av objekt med { _id: <ObjectId>, watercraftType: <ModelName> } för att gå över arrayen och sammanfoga vart och ett av dessa objekt med respektive samlingspost måste vi dela upp arrayen i individuella oberoende poster. $unwind funktionen skapar en ny datamängd för nästa sammanlagda steg

  { '$unwind': '$watercraftContexts' },

$unwind Resultat

Som du kan se $unwind skapar nu en post med en enda watercraftContext vi är nu inställda på att använda $lookup

[ { _id: ObjectId('5f7be2231da37c5b5915bf9b'),
    name: 'CAPTAIN_SAIL',
    classes: [ 'sail' ],
    license: 'WC-1',
    watercraftContexts:
     { _id: ObjectId('5f7be2231da37c5b5915bf8f'),
       watercraftType: 'Sailboat',
       ref: 'sailboats' },
    __v: 0 },
  { _id: ObjectId('5f7be2231da37c5b5915bf9c'),
    name: 'CAPTAIN_YATCH',
    classes: [ 'yatch' ],
    license: 'WC-2',
    watercraftContexts:
     { _id: ObjectId('5f7be2231da37c5b5915bf90'),
       watercraftType: 'Yatch',
       ref: 'yatches' },
    __v: 0 },
  { _id: ObjectId('5f7be2231da37c5b5915bf9d'),
    name: 'CAPTAIN_SHIP',
    classes: [ 'ship' ],
    license: 'WC-3',
    watercraftContexts:
     { _id: ObjectId('5f7be2231da37c5b5915bf91'),
       watercraftType: 'Ship',
       ref: 'ships' },
    __v: 0 },
  { _id: ObjectId('5f7be2231da37c5b5915bfa1'),
    name: 'CAPTAIN_SAIL_YATCH_SHIP',
    classes: [ 'sail', 'yatch', 'ship' ],
    license: 'WC-7',
    watercraftContexts:
     { _id: ObjectId('5f7be2231da37c5b5915bf98'),
       watercraftType: 'Sailboat',
       ref: 'sailboats' },
    __v: 0 },
  { _id: ObjectId('5f7be2231da37c5b5915bfa1'),
    name: 'CAPTAIN_SAIL_YATCH_SHIP',
    classes: [ 'sail', 'yatch', 'ship' ],
    license: 'WC-7',
    watercraftContexts:
     { _id: ObjectId('5f7be2231da37c5b5915bf99'),
       watercraftType: 'Yatch',
       ref: 'yatches' },
    __v: 0 },
  { _id: ObjectId('5f7be2231da37c5b5915bfa1'),
    name: 'CAPTAIN_SAIL_YATCH_SHIP',
    classes: [ 'sail', 'yatch', 'ship' ],
    license: 'WC-7',
    watercraftContexts:
     { _id: ObjectId('5f7be2231da37c5b5915bf9a'),
       watercraftType: 'Ship',
       ref: 'ships' },
    __v: 0 } ]

Steg 4 $lookup - Förenar varje post från den utländska samlingen

Det är viktigt att notera att vi måste $unwind innan du anropar $lookup för varje olika samling vi behöver gå med. Eftersom vi vill sammanfoga flera samlingar, måste vi lagra resultatet i ett objekt som angetts av samlingen för senare aggregering.

  // Only performs $lookup on 'ships' collection
  { '$lookup':
     { from: 'ships',  // Collection Name - Note: repeat for each collection
       localField: 'watercraftContexts._id', // The field with id to link
       foreignField: '_id',  // The field on the foreign collection to match
       as: 'watercrafts.ships' // The path where to store the lookup result
     }
  }

Steg 5 - Upprepa $unwind och $lookup för de andra joinningarna

Upprepa ovanstående till stegen för de ytterligare kopplingarna och knappa efter samlingens namn. Jag har kombinerat de samlade stadierna för att visa upprepningen.

  { '$unwind': '$watercraftContexts' },
  { '$lookup':
     { from: 'yatches',
       localField: 'watercraftContexts._id',
       foreignField: '_id',
       as: 'watercrafts.yatches' } },
  { '$unwind': '$watercraftContexts' },
  { '$lookup':
     { from: 'sailboats',
       localField: 'watercraftContexts._id',
       foreignField: '_id',
       as: 'watercrafts.sailboats' } }

Steg 4 &5 Resultat

Om du tittar noga märker du att en av Captain poster existerar 3 gånger med en annan watercraftType . $lookup returnerar endast poster som matchar specifika samlingsnamn. Det är därför du lagrar dem i ett Object knappad av collectionName

[
  { _id: ObjectId('5f7be7145320a65b942bb450'),
    name: 'CAPTAIN_SAIL',
    classes: [ 'sail' ],
    license: 'WC-1',
    watercraftContexts:
     { _id: ObjectId('5f7be7145320a65b942bb444'),
       watercraftType: 'Sailboat',
       ref: 'sailboats' },
    __v: 0,
    watercrafts:
     { ships: [],
       yatches: [],
       sailboats:
        [ { _id: ObjectId('5f7be7145320a65b942bb444'),
            class: 'sail',
            name: 'Gone with the Wind',
            __v: 0 } ] } },
  { _id: ObjectId('5f7be7145320a65b942bb451'),
    name: 'CAPTAIN_YATCH',
    classes: [ 'yatch' ],
    license: 'WC-2',
    watercraftContexts:
     { _id: ObjectId('5f7be7145320a65b942bb445'),
       watercraftType: 'Yatch',
       ref: 'yatches' },
    __v: 0,
    watercrafts:
     { ships: [],
       yatches:
        [ { _id: ObjectId('5f7be7145320a65b942bb445'),
            class: 'yatch',
            name: 'Liquid Gold',
            __v: 0 } ],
       sailboats: [] } },
  { _id: ObjectId('5f7be7145320a65b942bb452'),
    name: 'CAPTAIN_SHIP',
    classes: [ 'ship' ],
    license: 'WC-3',
    watercraftContexts:
     { _id: ObjectId('5f7be7145320a65b942bb446'),
       watercraftType: 'Ship',
       ref: 'ships' },
    __v: 0,
    watercrafts:
     { ships:
        [ { _id: ObjectId('5f7be7145320a65b942bb446'),
            class: 'ship',
            name: 'Jenny',
            __v: 0 } ],
       yatches: [],
       sailboats: [] } },
  { _id: ObjectId('5f7be7145320a65b942bb456'),
    name: 'CAPTAIN_SAIL_YATCH_SHIP',
    classes: [ 'sail', 'yatch', 'ship' ],
    license: 'WC-7',
    watercraftContexts:
     { _id: ObjectId('5f7be7145320a65b942bb44d'),
       watercraftType: 'Sailboat',
       ref: 'sailboats' },
    __v: 0,
    watercrafts:
     { ships: [],
       yatches: [],
       sailboats:
        [ { _id: ObjectId('5f7be7145320a65b942bb44d'),
            class: 'sail',
            name: 'Swell Shredder',
            __v: 0 } ] } },
  { _id: ObjectId('5f7be7145320a65b942bb456'),
    name: 'CAPTAIN_SAIL_YATCH_SHIP',
    classes: [ 'sail', 'yatch', 'ship' ],
    license: 'WC-7',
    watercraftContexts:
     { _id: ObjectId('5f7be7145320a65b942bb44e'),
       watercraftType: 'Yatch',
       ref: 'yatches' },
    __v: 0,
    watercrafts:
     { ships: [],
       yatches:
        [ { _id: ObjectId('5f7be7145320a65b942bb44e'),
            class: 'yatch',
            name: 'Audrey',
            __v: 0 } ],
       sailboats: [] } },
  { _id: ObjectId('5f7be7145320a65b942bb456'),
    name: 'CAPTAIN_SAIL_YATCH_SHIP',
    classes: [ 'sail', 'yatch', 'ship' ],
    license: 'WC-7',
    watercraftContexts:
     { _id: ObjectId('5f7be7145320a65b942bb44f'),
       watercraftType: 'Ship',
       ref: 'ships' },
    __v: 0,
    watercrafts:
     { ships:
        [ { _id: ObjectId('5f7be7145320a65b942bb44f'),
            class: 'ship',
            name: 'Jenny IV',
            __v: 0 } ],
       yatches: [],
       sailboats: [] } } ]

Steg 6 $project - Använd project för att platta till objektkartan över sammanfogningar

Vi kan använda Project för att välja all befintlig data och förenkla objektkartan över sammanfogningsresultat till en enda Array.

  { '$project':
     // keys with the value 'true' will be included
     { name: true,
       license: true,
       classes: true,
       _id: true,
       watercraftContexts: true,
       __v: true,
       watercrafts:            // Re-assigns value of watercrafts
        { '$setUnion':         // Accepts an array of arrays to flatten
           [
             '$watercrafts.ships',
             '$watercrafts.yatches',
             '$watercrafts.sailboats'
           ]
        }
     }
  }

$project Resultat

Resultaten av ovanstående $project kommer att ersätta watercrafts objekt med en platt array av watercrafts , men det är viktigt att notera att det fortfarande finns dubbletter av Captain där matchar många olika uppslag. Vi sätter ihop dem igen i nästa steg.

[ { _id: ObjectId('5f7bea8d79dfe25bf3cb9695'),
    name: 'CAPTAIN_SAIL',
    classes: [ 'sail' ],
    license: 'WC-1',
    watercraftContexts:
     { _id: ObjectId('5f7bea8d79dfe25bf3cb9689'),
       watercraftType: 'Sailboat',
       ref: 'sailboats' },
    __v: 0,
    watercrafts:
     [ { _id: ObjectId('5f7bea8d79dfe25bf3cb9689'),
         class: 'sail',
         name: 'Gone with the Wind',
         __v: 0 } ] },
  { _id: ObjectId('5f7bea8d79dfe25bf3cb9696'),
    name: 'CAPTAIN_YATCH',
    classes: [ 'yatch' ],
    license: 'WC-2',
    watercraftContexts:
     { _id: ObjectId('5f7bea8d79dfe25bf3cb968a'),
       watercraftType: 'Yatch',
       ref: 'yatches' },
    __v: 0,
    watercrafts:
     [ { _id: ObjectId('5f7bea8d79dfe25bf3cb968a'),
         class: 'yatch',
         name: 'Liquid Gold',
         __v: 0 } ] },
  { _id: ObjectId('5f7bea8d79dfe25bf3cb9697'),
    name: 'CAPTAIN_SHIP',
    classes: [ 'ship' ],
    license: 'WC-3',
    watercraftContexts:
     { _id: ObjectId('5f7bea8d79dfe25bf3cb968b'),
       watercraftType: 'Ship',
       ref: 'ships' },
    __v: 0,
    watercrafts:
     [ { _id: ObjectId('5f7bea8d79dfe25bf3cb968b'),
         class: 'ship',
         name: 'Jenny',
         __v: 0 } ] },
  { _id: ObjectId('5f7bea8d79dfe25bf3cb969b'),
    name: 'CAPTAIN_SAIL_YATCH_SHIP',
    classes: [ 'sail', 'yatch', 'ship' ],
    license: 'WC-7',
    watercraftContexts:
     { _id: ObjectId('5f7bea8d79dfe25bf3cb9692'),
       watercraftType: 'Sailboat',
       ref: 'sailboats' },
    __v: 0,
    watercrafts:
     [ { _id: ObjectId('5f7bea8d79dfe25bf3cb9692'),
         class: 'sail',
         name: 'Swell Shredder',
         __v: 0 } ] },
  { _id: ObjectId('5f7bea8d79dfe25bf3cb969b'),
    name: 'CAPTAIN_SAIL_YATCH_SHIP',
    classes: [ 'sail', 'yatch', 'ship' ],
    license: 'WC-7',
    watercraftContexts:
     { _id: ObjectId('5f7bea8d79dfe25bf3cb9693'),
       watercraftType: 'Yatch',
       ref: 'yatches' },
    __v: 0,
    watercrafts:
     [ { _id: ObjectId('5f7bea8d79dfe25bf3cb9693'),
         class: 'yatch',
         name: 'Audrey',
         __v: 0 } ] },
  { _id: ObjectId('5f7bea8d79dfe25bf3cb969b'),
    name: 'CAPTAIN_SAIL_YATCH_SHIP',
    classes: [ 'sail', 'yatch', 'ship' ],
    license: 'WC-7',
    watercraftContexts:
     { _id: ObjectId('5f7bea8d79dfe25bf3cb9694'),
       watercraftType: 'Ship',
       ref: 'ships' },
    __v: 0,
    watercrafts:
     [ { _id: ObjectId('5f7bea8d79dfe25bf3cb9694'),
         class: 'ship',
         name: 'Jenny IV',
         __v: 0 } ] } ]

Steg 7 $unwind and $group

Vi $unwind så att vi nu kan gruppera alla watercrafts som tillhör samma Captain . Vi måste också använda $mergeObjects för att tillfälligt lagra ytterligare data från Captain insamling under en ny temporär variabel för att förbereda sig för slutskedet.

  { '$unwind': '$watercrafts' },
  { '$group':
     { _id: '$_id',
       data:
        { '$mergeObjects':
           { name: '$name',
             license: '$license',
             classes: '$classes',
             watercraftContexts: '$watercraftContexts',
             __v: '$__v' } },
       watercrafts: { '$push': '$watercrafts' } } }

$unwind och $group Resultat

Nu är vi verkligen på väg någonstans. Vi har reducerat vår transformation till vår ursprungliga 4 Captain s och plattade våra sammanfogningar till en enda array.

[ { _id: ObjectId('5f7bed5e271dd95c306c25a4'),
    data:
     { name: 'CAPTAIN_SHIP',
       license: 'WC-3',
       classes: [ 'ship' ],
       watercraftContexts:
        { _id: ObjectId('5f7bed5e271dd95c306c2598'),
          watercraftType: 'Ship',
          ref: 'ships' },
       __v: 0 },
    watercrafts:
     [ { _id: ObjectId('5f7bed5e271dd95c306c2598'),
         class: 'ship',
         name: 'Jenny',
         __v: 0 } ] },
  { _id: ObjectId('5f7bed5e271dd95c306c25a8'),
    data:
     { name: 'CAPTAIN_SAIL_YATCH_SHIP',
       license: 'WC-7',
       classes: [ 'sail', 'yatch', 'ship' ],
       watercraftContexts:
        { _id: ObjectId('5f7bed5e271dd95c306c25a1'),
          watercraftType: 'Ship',
          ref: 'ships' },
       __v: 0 },
    watercrafts:
     [ { _id: ObjectId('5f7bed5e271dd95c306c259f'),
         class: 'sail',
         name: 'Swell Shredder',
         __v: 0 },
       { _id: ObjectId('5f7bed5e271dd95c306c25a0'),
         class: 'yatch',
         name: 'Audrey',
         __v: 0 },
       { _id: ObjectId('5f7bed5e271dd95c306c25a1'),
         class: 'ship',
         name: 'Jenny IV',
         __v: 0 } ] },
  { _id: ObjectId('5f7bed5e271dd95c306c25a2'),
    data:
     { name: 'CAPTAIN_SAIL',
       license: 'WC-1',
       classes: [ 'sail' ],
       watercraftContexts:
        { _id: Object('5f7bed5e271dd95c306c2596'),
          watercraftType: 'Sailboat',
          ref: 'sailboats' },
       __v: 0 },
    watercrafts:
     [ { _id: ObjectId('5f7bed5e271dd95c306c2596'),
         class: 'sail',
         name: 'Gone with the Wind',
         __v: 0 } ] },
  { _id: ObjectId('5f7bed5e271dd95c306c25a3'),
    data:
     { name: 'CAPTAIN_YATCH',
       license: 'WC-2',
       classes: [ 'yatch' ],
       watercraftContexts:
        { _id: ObjectId('5f7bed5e271dd95c306c2597'),
          watercraftType: 'Yatch',
          ref: 'yatches' },
       __v: 0 },
    watercrafts:
     [ { _id: ObjectId('5f7bed5e271dd95c306c2597'),
         class: 'yatch',
         name: 'Liquid Gold',
         __v: 0 } ] } ]

Steg 8 $replaceRoot och $project

Allt vi har kvar är att slå samman våra data i roten för varje post och ta bort den temporära variabeln data

  // Merges 'data' into the root of each record
  { '$replaceRoot': { newRoot: { '$mergeObjects': [ '$data', '$$ROOT' ] } } },
  // Use $project to remove data (include only the fields we want)
  { '$project':
     { name: true,
       license: true,
       classes: true,
       _id: true,
       watercraftContexts: true,
       __v: true,
       watercrafts: true } 
  }

$replaceRoot &$project Resultat

Nu har vi resultatet vi satt ut för...En Captain med en rad blandade associerade typer watercrafts

[ 
  { name: 'CAPTAIN_SAIL_YATCH_SHIP',
    license: 'WC-7',
    classes: [ 'sail', 'yatch', 'ship' ],
    watercraftContexts:
     { _id: ObjectId('5f7bf3b3680b375ca1755ea6'),
       watercraftType: 'Ship',
       ref: 'ships' },
    __v: 0,
    _id: ObjectId('5f7bf3b3680b375ca1755ead'),
    watercrafts:
     [ { _id: ObjectId('5f7bf3b3680b375ca1755ea4'),
         class: 'sail',
         name: 'Swell Shredder',
         __v: 0 },
       { _id: ObjectId('5f7bf3b3680b375ca1755ea5'),
         class: 'yatch',
         name: 'Audrey',
         __v: 0 },
       { _id: ObjectId('5f7bf3b3680b375ca1755ea6'),
         class: 'ship',
         name: 'Jenny IV',
         __v: 0 } ] },
  { name: 'CAPTAIN_SAIL',
    license: 'WC-1',
    classes: [ 'sail' ],
    watercraftContexts:
     { _id: ObjectId('5f7bf3b3680b375ca1755e9b'),
       watercraftType: 'Sailboat',
       ref: 'sailboats' },
    __v: 0,
    _id: ObjectId('5f7bf3b3680b375ca1755ea7'),
    watercrafts:
     [ { _id: ObjectId('5f7bf3b3680b375ca1755e9b'),
         class: 'sail',
         name: 'Gone with the Wind',
         __v: 0 } ] },
  { name: 'CAPTAIN_YATCH',
    license: 'WC-2',
    classes: [ 'yatch' ],
    watercraftContexts:
     { _id: ObjectId('5f7bf3b3680b375ca1755e9c'),
       watercraftType: 'Yatch',
       ref: 'yatches' },
    __v: 0,
    _id: ObjectId('5f7bf3b3680b375ca1755ea8'),
    watercrafts:
     [ { _id: ObjectId('5f7bf3b3680b375ca1755e9c'),
         class: 'yatch',
         name: 'Liquid Gold',
         __v: 0 } ] },
  { name: 'CAPTAIN_SHIP',
    license: 'WC-3',
    classes: [ 'ship' ],
    watercraftContexts:
     { _id: ObjectId('5f7bf3b3680b375ca1755e9d'),
       watercraftType: 'Ship',
       ref: 'ships' },
    __v: 0,
    _id: ObjectId('5f7bf3b3680b375ca1755ea9'),
    watercrafts:
     [ { _id: ObjectId('5f7bf3b3680b375ca1755e9d'),
         class: 'ship',
         name: 'Jenny',
         __v: 0 } ] } ]

Och där har du det... tog bara 2 dagar att lista ut detta. Jag hoppas att det sparar dig lite tid om du försöker med en liknande sammanlagd association. Lycka till med kodningen!

Slutlig pipeline

[ 
  { '$match':
     { _id:
        { '$in':
           [ ObjectId('5f7bf3b3680b375ca1755ea9'),
             ObjectId('5f7bf3b3680b375ca1755ea7'),
             ObjectId('5f7bf3b3680b375ca1755ea8'),
             ObjectId('5f7bf3b3680b375ca1755ead')
           ]
        }
     }
  },
  { '$unwind': '$watercraftContexts' },
  { '$lookup':
     { from: 'ships',
       localField: 'watercraftContexts._id',
       foreignField: '_id',
       as: 'watercrafts.ships' } },
  { '$unwind': '$watercraftContexts' },
  { '$lookup':
     { from: 'yatches',
       localField: 'watercraftContexts._id',
       foreignField: '_id',
       as: 'watercrafts.yatches' } },
  { '$unwind': '$watercraftContexts' },
  { '$lookup':
     { from: 'sailboats',
       localField: 'watercraftContexts._id',
       foreignField: '_id',
       as: 'watercrafts.sailboats' } },
  { '$project':
     { name: true,
       license: true,
       classes: true,
       _id: true,
       watercraftContexts: true,
       __v: true,
       watercrafts:
        { '$setUnion':
           [ '$watercrafts.ships',
             '$watercrafts.yatches',
             '$watercrafts.sailboats' ] } } },
  { '$unwind': '$watercrafts' },
  { '$group':
     { _id: '$_id',
       data:
        { '$mergeObjects':
           { name: '$name',
             license: '$license',
             classes: '$classes',
             watercraftContexts: '$watercraftContexts',
             __v: '$__v' } },
       watercrafts: { '$push': '$watercrafts' } } },
  { '$replaceRoot': { newRoot: { '$mergeObjects': [ '$data', '$$ROOT' ] } } },
  { '$project':
     { name: true,
       license: true,
       classes: true,
       _id: true,
       watercraftContexts: true,
       __v: true,
       watercrafts: true } }
]