Arpit Bhayani Blogs

336 files168 subfolders

Shared Workspace

Arpit Bhayani Blogs

001 Ai Topological Sort

tldr

Shared from "Arpit Bhayani Blogs" on Inkdown

TLDR: MongoDB Pagination - Two Approaches You Need to Know

Date: 2017-03-22 Source: https://arpitbhayani.me/blogs/fast-and-efficient-pagination-in-mongodb

Overview

Learn how to efficiently paginate MongoDB data using skip/limit and _id-based methods. Improve performance for batch processing & UI display. MongoDB is a document based data store and hence pagination is one of the most common use case of it.

Key Points

Showing huge set of results on user interface
Approach 1: Using cursor.skip and cursor.limit: MongoDB cursor has two methods that makes paging easy; they are skip(n) will skip n documents from the cursor while limit(n) will cap the number of documents to be returned from the cursor.

001-ai-topological-sort.md

002 Temporal Primer

002-temporal-primer.md

003 Rag Production

003-rag-production.md

004 Structure Of Llm Chat

004-structure-of-llm-chat.md

005 How Llms Work

005-how-llms-work.md

006 Monolith Is Distributed System

006-monolith-is-distributed-system.md

007 Defensive Databases

007-defensive-databases.md

008 Bm25

009 Join Algorithms

009-join-algorithms.md

010 Venting At Work

010-venting-at-work.md

011 Half Life

011-half-life.md

012 Multi Paxos

012-multi-paxos.md

013 Mysql Replication Internals

013-mysql-replication-internals.md

014 Bloom Filters

014-bloom-filters.md

015 Clock Sync Nightmare

015-clock-sync-nightmare.md

016 Kafka Partitions

016-kafka-partitions.md

017 Product Quantization

017-product-quantization.md

018 Qkv Matrices

018-qkv-matrices.md

019 Deleted Production

019-deleted-production.md

020 How Llm Inference Works

020-how-llm-inference-works.md

021 Blocking Queues

021-blocking-queues.md

022 Heartbeats In Distributed Systems

022-heartbeats-in-distributed-systems.md

023 Cassandra Writes

023-cassandra-writes.md

024 Redis Replication

024-redis-replication.md

025 Arrogant People At Work

025-arrogant-people-at-work.md

026 Cdn Content Replication

026-cdn-content-replication.md

027 Cant Fix Everything Day One

027-cant-fix-everything-day-one.md

028 Emotions At Work

028-emotions-at-work.md

029 Grpc Http2

029-grpc-http2.md

030 Meetings With No Agenda Are A Waste Of Time

030-meetings-with-no-agenda-are-a-waste-of-time.md

031 Growth Is Not About Doing Everything

031-growth-is-not-about-doing-everything.md

032 Career Longevity Vs Job Hopping

032-career-longevity-vs-job-hopping.md

033 Stay Relevant At Higher Salary Levels

033-stay-relevant-at-higher-salary-levels.md

034 Why Consensus

034-why-consensus.md

035 Database Deadlocks

035-database-deadlocks.md

036 Cpu Cache Locality

036-cpu-cache-locality.md

037 Eventual Consistency

037-eventual-consistency.md

038 Dns Udp Tcp

038-dns-udp-tcp.md

039 Masters

040 Empathy Makes Great Engineers Unstoppable

040-empathy-makes-great-engineers-unstoppable.md

041 Good Mentors Build People

041-good-mentors-build-people.md

042 Always Have Back Burner Projects

042-always-have-back-burner-projects.md

043 Before You Push Back Know What Youre Standing On

043-before-you-push-back-know-what-youre-standing-on.md

044 Be The One They Can Count On

044-be-the-one-they-can-count-on.md

045 How Much People Bet On You

045-how-much-people-bet-on-you.md

046 How To Get Leadership To Say Yes To Your Project

046-how-to-get-leadership-to-say-yes-to-your-project.md

047 Dont Let Your Best Ideas Die In Silence

047-dont-let-your-best-ideas-die-in-silence.md

048 Be Someone Others Want To Work With

048-be-someone-others-want-to-work-with.md

049 Dont Fall For Xy Problem Ask Right Questions

049-dont-fall-for-xy-problem-ask-right-questions.md

050 Biggest Lie Startups Tell Engineers

050-biggest-lie-startups-tell-engineers.md

051 Promotions Are Proactive Not Reactive

051-promotions-are-proactive-not-reactive.md

052 Not Enough To Be Right Learn To Be Heard

052-not-enough-to-be-right-learn-to-be-heard.md

053 No One Ships Alone

053-no-one-ships-alone.md

054 Not Every Mistake Needs A Correction

054-not-every-mistake-needs-a-correction.md

055 Build Influence At Work

055-build-influence-at-work.md

056 Your Soft Skills Arent Soft At All

056-your-soft-skills-arent-soft-at-all.md

057 Experience Before Forming Opinion

057-experience-before-forming-opinion.md

058 Curiosity And High Bias For Action

058-curiosity-and-high-bias-for-action.md

059 Worklog

060 Mistakes And Growth

060-mistakes-and-growth.md

061 Own It Instead Of Sweeping It Aside

061-own-it-instead-of-sweeping-it-aside.md

062 Dont Wait Step Up

062-dont-wait-step-up.md

063 Temporary Fix Is Permanent

063-temporary-fix-is-permanent.md

064 Interview Bias And What Sets You Apart

064-interview-bias-and-what-sets-you-apart.md

065 Saying This Isnt My Problem Is A Problem

065-saying-this-isnt-my-problem-is-a-problem.md

066 Okr

067 Miscommunication

067-miscommunication.md

068 When In Doubt Code It Out

068-when-in-doubt-code-it-out.md

069 Follow Up Without Annoying People

069-follow-up-without-annoying-people.md

070 Lead Projects That Land

070-lead-projects-that-land.md

071 Abstract Thinking Skill Next Decade

071-abstract-thinking-skill-next-decade.md

072 We Engineers Suck At Task Estimation

072-we-engineers-suck-at-task-estimation.md

073 Shiny Object Syndrome In Tech

073-shiny-object-syndrome-in-tech.md

074 3p

075 Leverage The Equilibrium

075-leverage-the-equilibrium.md

076 On Demand Container Loading In Aws Lambda

076-on-demand-container-loading-in-aws-lambda.md

077 Sql Has Problems We Can Fix Them Pipe Syntax In Sql

077-sql-has-problems-we-can-fix-them-pipe-syntax-in-sql.md

078 Nanolog A Nanosecond Scale Logging System

078-nanolog-a-nanosecond-scale-logging-system.md

079 Best Resource Is Mythical

079-best-resource-is-mythical.md

080 Wtf The Who To Follow Service At Twitter

080-wtf-the-who-to-follow-service-at-twitter.md

081 Know A Lot

081-know-a-lot.md

082 Out Of Syllabus

082-out-of-syllabus.md

083 Negotiate The Offer

083-negotiate-the-offer.md

084 Never Bad Mouth Your Ex Exployer

084-never-bad-mouth-your-ex-exployer.md

085 Culture Fit

085-culture-fit.md

086 Quantification In Resume

086-quantification-in-resume.md

087 Hiring Is Unfair

087-hiring-is-unfair.md

088 Questions For Interviewers

088-questions-for-interviewers.md

089 Collaboration Communication

089-collaboration-communication.md

090 Out Of Vicious Interview Cycle

090-out-of-vicious-interview-cycle.md

091 Pitch Projects Not Ideas

091-pitch-projects-not-ideas.md

092 Read Design Docs

092-read-design-docs.md

093 Read Rca Docs

093-read-rca-docs.md

094 Start Generalist

094-start-generalist.md

095 Do Not Rely On Summaries

095-do-not-rely-on-summaries.md

096 Structure Your Design Interviews

096-structure-your-design-interviews.md

097 Title Inflation

097-title-inflation.md

098 Find Your Own Project

098-find-your-own-project.md

099 Six Pointers To Crack Coding And Design Interviews

099-six-pointers-to-crack-coding-and-design-interviews.md

100 Keep Yourself Unblocked

100-keep-yourself-unblocked.md

101 Genetic Knapsack

101-genetic-knapsack.md

102 Pseudorandom Number Generation Lfsr

102-pseudorandom-number-generation-lfsr.md

103 How Indexes Work On Partitioned And Sharded Data

103-how-indexes-work-on-partitioned-and-sharded-data.md

104 Some Data Partitioning Strategies For Distributed Data Stores

104-some-data-partitioning-strategies-for-distributed-data-stores.md

105 Data Partitioning

105-data-partitioning.md

106 Leaderless Replication

106-leaderless-replication.md

107 Conflict Resolution

107-conflict-resolution.md

108 Conflict Detection

108-conflict-detection.md

109 Multi Master Replication

109-multi-master-replication.md

110 Monotonic Reads

110-monotonic-reads.md

111 Read Your Write Consistency

111-read-your-write-consistency.md

112 Handling Outages Master Replica

112-handling-outages-master-replica.md

113 Replication Formats

113-replication-formats.md

114 Replication Strategies

114-replication-strategies.md

115 Master Replica Replication

115-master-replica-replication.md

116 Durability

116-durability.md

117 Isolation

117-isolation.md

118 Atomicity

118-atomicity.md

119 Consistency

119-consistency.md

120 Architectures In Distributed Systems

120-architectures-in-distributed-systems.md

121 Mistaken Beliefs Of Distributed Systems

121-mistaken-beliefs-of-distributed-systems.md

122 Fork Bomb

122-fork-bomb.md

123 Chained Operators Python

123-chained-operators-python.md

124 Taxonomy On Sql

124-taxonomy-on-sql.md

125 The Weird Walrus

125-the-weird-walrus.md

126 Fully Persistent Arrays

126-fully-persistent-arrays.md

127 Persistent Data Structures Introduction

127-persistent-data-structures-introduction.md

128 Constant Folding Python

128-constant-folding-python.md

129 String Interning Python

129-string-interning-python.md

130 Recursion Visualizer Python

130-recursion-visualizer-python.md

131 Flajolet Martin

131-flajolet-martin.md

132 2q Cache

132-2q-cache.md

133 Israeli Queues

133-israeli-queues.md

134 1d Terrain

134-1d-terrain.md

135 Jaccard Minhash

135-jaccard-minhash.md

136 Ts Smoothing

136-ts-smoothing.md

137 Lfu

138 Morris Counter

138-morris-counter.md

139 Slowsort

139-slowsort.md

140 Bitcask

141 Phi Accrual

141-phi-accrual.md

142 10x Engineer

142-10x-engineer.md

143 Decipher Repeated Key Xor

143-decipher-repeated-key-xor.md

144 Decipher Single Xor

144-decipher-single-xor.md

145 Python Iterable Integers

145-python-iterable-integers.md

146 Inheritance C

146-inheritance-c.md

147 Rum

148 Consistent Hashing

148-consistent-hashing.md

149 Python Caches Integers

149-python-caches-integers.md

150 Fractional Cascading

150-fractional-cascading.md

151 Copy On Write

151-copy-on-write.md

152 Midpoint Insertion Caching Strategy

152-midpoint-insertion-caching-strategy.md

153 Fsm Python

153-fsm-python.md

154 Bayesian Average

154-bayesian-average.md

155 Sliding Window Ratelimiter

155-sliding-window-ratelimiter.md

156 Idf

157 Better Programmer

157-better-programmer.md

158 Python Prompts

158-python-prompts.md

159 Rule 30 Cellular Automata

159-rule-30-cellular-automata.md

160 Function Overloading

160-function-overloading.md

161 Isolation Forest

161-isolation-forest.md

162 Image Steganography

162-image-steganography.md

163 Long Integers Python

163-long-integers-python.md

164 I Changed My Python

164-i-changed-my-python.md

165 Benchmark And Compare Pagination Approach In Mongodb

165-benchmark-and-compare-pagination-approach-in-mongodb.md

166 Mongodb Cursor Skip Is Slow

166-mongodb-cursor-skip-is-slow.md

167 Fast And Efficient Pagination In Mongodb

167-fast-and-efficient-pagination-in-mongodb.md

168 Making Http Requests Using Netcat

168-making-http-requests-using-netcat.md

Approach 2: Using _id and limit: This approach will make effective use of default index on _id and nature of ObjectId.

Closing Remarks: Both of the above approaches are valid and correct.

Takeaway

Both of the above approaches are valid and correct.