What is Python and what are its primary characteristics?
Python is a high-level, interpreted programming language created by Guido van Rossum. It is known for its simplicity, readability, and versatility, with key characteristics including dynamic typing, automatic memory management, and support for multiple programming paradigms.
Explain the difference between Python 2 and Python 3.
Python 3 is a major revision of the language that is not fully backward-compatible with Python 2. Key differences include print function vs. print statement, integer division behavior, Unicode string handling, and removal of certain legacy syntax.
What is the Global Interpreter Lock (GIL) in Python?
The Global Interpreter Lock is a mutex that prevents multiple native threads from executing Python bytecodes simultaneously. It's a mechanism in CPython that ensures thread safety but can limit the performance of multi-threaded programs, especially in CPU-bound tasks.
What are the main programming paradigms supported by Python?
Python supports multiple programming paradigms, including object-oriented programming (OOP), functional programming, and procedural programming. It allows developers to use different programming styles based on their requirements.
How does Python's memory management work?
Python uses automatic memory management with reference counting and garbage collection. Objects are created dynamically, and memory is automatically allocated and freed. The reference count of an object is incremented when it's assigned to a variable and decremented when references are removed.
Explain the concept of duck typing in Python.
Duck typing is a dynamic typing concept in Python where the type or class of an object is less important than the methods it defines. If an object has all the methods and properties required for a particular use, it can be used, regardless of its actual type.
What is the difference between .py and .pyc files?
.py files are source code files containing Python script, while .pyc files are compiled bytecode files that are created to improve performance by reducing compilation time in subsequent runs.
What are decorators in Python?
Decorators are a design pattern in Python that allows modifying or enhancing functions or classes without directly changing their source code. They are functions that take another function as an argument and return a modified version of that function.
Describe Python's method resolution order (MRO).
Method Resolution Order (MRO) is the order in which Python looks for methods in a hierarchy of classes. It uses the C3 linearization algorithm to determine the order of method resolution in multiple inheritance scenarios, ensuring a consistent and predictable method lookup.
What is PEP 8 and why is it important?
PEP 8 is the style guide for Python code that provides conventions for writing readable and consistent Python code. It covers aspects like indentation, naming conventions, maximum line length, and other coding standards to improve code readability and maintainability.
Explain the difference between lists and tuples in Python.
Lists are mutable, ordered collections that can be modified after creation, while tuples are immutable and cannot be changed once created. Lists use square brackets [], tuples use parentheses (), and tuples are generally more memory-efficient and can be used as dictionary keys.
What are lambda functions in Python?
Lambda functions are small, anonymous functions defined using the 'lambda' keyword. They can have any number of arguments but can only have one expression. They are often used for short, one-time use functions, especially as arguments to higher-order functions.
How does Python's 'with' statement work?
The 'with' statement is used for resource management, ensuring that a resource is properly acquired and released. It simplifies exception handling by automatically calling close() or exit() methods, making code cleaner and reducing the risk of resource leaks.
What is the difference between deep and shallow copy?
A shallow copy creates a new object but references the same memory addresses for nested objects, while a deep copy creates a completely independent copy of an object and all its nested objects. The copy module provides methods for both types of copying.
Explain Python's namespaces and scoping rules.
Python uses namespaces to organize and manage variable names. It follows the LEGB rule (Local, Enclosing, Global, Built-in) for variable scope resolution. Each namespace is a mapping from names to objects, and Python searches for names in this specific order.
What are generator functions in Python?
Generator functions use the 'yield' keyword to return a generator iterator. They allow you to generate values on-the-fly and pause the function's state, making them memory-efficient for handling large datasets or infinite sequences.
What is the purpose of the __init__ method in Python classes?
The __init__ method is a special constructor method in Python classes that is automatically called when a new object is created. It initializes the object's attributes and can take parameters to set up the initial state of the object.
How does Python handle multiple inheritance?
Python supports multiple inheritance, allowing a class to inherit from multiple parent classes. It uses the Method Resolution Order (MRO) to determine the order of method inheritance, resolving potential conflicts using the C3 linearization algorithm.
What are context managers in Python?
Context managers are objects that define __enter__ and __exit__ methods, used with the 'with' statement to manage resources. They provide a clean way to set up and tear down resources, ensuring proper initialization and cleanup.
Explain the difference between == and is operators in Python.
The '==' operator checks for value equality, comparing the values of two objects, while the 'is' operator checks for identity, verifying if two references point to the exact same object in memory.
What are metaclasses in Python?
Metaclasses are classes of classes, allowing you to customize class creation. They define how a class behaves and can modify the class definition at runtime. The type class is the default metaclass in Python.
How do you handle exceptions in Python?
Python uses try-except blocks to handle exceptions. You can catch specific exceptions, use multiple except blocks, include an else clause for code to run if no exception occurs, and use a finally clause for cleanup code that always runs.
What is the purpose of the __str__ and __repr__ methods?
__str__ provides a human-readable string representation of an object, typically used by print(), while __repr__ returns a more detailed, unambiguous representation often used for debugging and development.
Explain Python's garbage collection mechanism.
Python uses reference counting and generational garbage collection. When an object's reference count drops to zero, it's immediately deallocated. For circular references, a cyclic garbage collector periodically runs to identify and remove unreachable objects.
What are magic methods (dunder methods) in Python?
Magic methods, identified by double underscores (e.g., __init__, __str__), allow customization of object behavior. They define how objects interact with built-in functions and operators, enabling operator overloading and custom object implementations.
How does Python support functional programming?
Python supports functional programming through features like lambda functions, map(), filter(), reduce(), comprehensions, and first-class functions. These allow for writing code in a more declarative and concise manner.
What is the difference between abstract base classes and interfaces in Python?
Abstract base classes (defined using the abc module) can have both abstract and concrete methods, while Python doesn't have traditional interfaces. Abstract base classes provide a way to define a blueprint for other classes, ensuring certain methods are implemented.
Explain type hinting and its benefits in Python.
Type hinting allows specifying expected types for function parameters and return values. Introduced in Python 3.5, it improves code readability, helps catch type-related errors early, and enables better IDE support and static type checking.
What are *args and **kwargs in Python functions?
*args allows a function to accept any number of positional arguments, while **kwargs allows accepting any number of keyword arguments. They provide flexibility in function definitions and are commonly used in wrapper functions and class inheritance.
How does Python implement data encapsulation?
Python uses naming conventions for encapsulation. Single underscore prefix (e.g., _variable) suggests internal use, double underscore prefix (e.g., __variable) triggers name mangling to prevent naming conflicts in inheritance.
What is the difference between lists and tuples in Python?
Lists are mutable (can be modified after creation) and use square brackets [], while tuples are immutable (cannot be modified after creation) and use parentheses (). Lists have methods like append(), remove(), while tuples have limited methods due to immutability.
Explain the difference between '==' and 'is' operators in Python.
'==' compares the values of objects (equality), while 'is' compares the identity (memory location) of objects. For example, a == b checks if a and b have the same value, while a is b checks if they are the same object in memory.
What are the different numeric data types in Python?
Python has int (integer), float (floating-point), complex (complex numbers) as numeric types. Integers have unlimited precision, floats are typically double-precision, and complex numbers have real and imaginary parts.
How does Python handle variable scoping?
Python uses LEGB rule: Local, Enclosing, Global, Built-in. Variables are first searched in local scope, then enclosing functions, then global scope, and finally built-in scope. Use 'global' keyword to modify global variables from local scope.
What is list comprehension and what are its advantages?
List comprehension is a concise way to create lists based on existing lists/iterables. Syntax: [expression for item in iterable if condition]. Advantages include more readable and compact code, better performance than equivalent for loops, and clearer intent.
Explain slicing in Python and its extended form.
Slicing extracts parts of sequences using syntax sequence[start:stop:step]. Start is inclusive, stop is exclusive. Negative indices count from end. Extended form allows step value to control direction and skipping. Default step is 1.
What are Python's string formatting methods?
Python offers multiple string formatting methods: %-formatting (old style), str.format() method, and f-strings (3.6+). F-strings are most readable: f'{variable}'. Format() uses {}, %-formatting uses %d, %s etc. Each has different use cases and syntax.
How does Python's garbage collection work?
Python uses reference counting for garbage collection, plus a cyclic garbage collector for circular references. When an object's reference count reaches zero, it's deallocated. The cyclic collector identifies and collects unreachable reference cycles periodically.
What are the different types of operators in Python?
Python includes arithmetic (+, -, *, /, //, %, **), comparison (==, !=, >, <, >=, <=), logical (and, or, not), bitwise (&, |, ^, ~, <<, >>), assignment (=, +=, -=, etc.), and identity/membership (is, is not, in, not in) operators.
How do you handle type hints in Python?
Type hints are added using annotations: function parameters (param: type), return types (-> type), and variable annotations (var: type). Use typing module for complex types. They're hints only, not enforced at runtime by default.
What are generator expressions and when should they be used?
Generator expressions create iterators using syntax (expression for item in iterable if condition). Similar to list comprehension but more memory efficient for large datasets as they generate values on-the-fly instead of storing all at once.
Explain the difference between deep and shallow copy.
Shallow copy creates new object but references same nested objects (copy.copy()). Deep copy creates new object and recursively copies nested objects (copy.deepcopy()). Choose based on whether nested objects need independent copies.
What are magic methods (dunder methods) in Python?
Magic methods are special methods with double underscores (e.g., __init__, __str__) that define behavior for operations like initialization, string representation, comparison, arithmetic. They customize class behavior and operator overloading.
How does Python handle memory management?
Python uses private heap space for memory management. Memory manager allocates heap space for objects. Reference counting tracks object usage. Memory pool for small objects improves allocation speed. Garbage collector frees unused memory.
What are the different ways to handle conditional execution?
Python offers if/elif/else statements, ternary operators (value_if_true if condition else value_if_false), and conditional expressions. Also supports match/case (3.10+) for pattern matching and boolean short-circuit evaluation.
How do you use walrus operator (:=) and what are its benefits?
Walrus operator assigns values in expressions: while (n := len(a)) > 0. Benefits include more concise code, avoiding repetitive expressions, and combining assignment with conditional checks. Introduced in Python 3.8.
What are the different loop control statements in Python?
Python provides break (exit loop), continue (skip to next iteration), else (executes when loop completes normally), and pass (null statement) for loop control. Can be used in both for and while loops.
How does Python handle variable assignment and reference?
Python assigns references to objects, not objects themselves. Multiple names can reference same object. Assignment creates new reference, not copy. Understanding this is crucial for mutable vs immutable objects behavior.
What are sequence unpacking and packing in Python?
Unpacking assigns sequence elements to variables: a, b = [1, 2]. Packing collects multiple values into sequence using *args for positional and **kwargs for keyword arguments. Supports extended unpacking with * operator.
How do you handle string encoding and decoding in Python?
Use encode() to convert string to bytes, decode() for bytes to string. Specify encoding (e.g., 'utf-8', 'ascii'). Handle encoding errors with error handlers like 'ignore', 'replace', 'strict'. Important for file I/O and network operations.
What are the differences between range, xrange (Python 2), and range (Python 3)?
Python 2's range creates list, xrange creates iterator. Python 3's range creates range object (iterator-like). Range object more memory efficient as it generates values on demand rather than storing all in memory.
How do you handle boolean operations and short-circuit evaluation?
Boolean operations (and, or, not) follow short-circuit rules. 'and' returns first false value or last value, 'or' returns first true value or last value. Short-circuiting optimizes evaluation by skipping unnecessary checks.
What are f-strings and their advanced features?
F-strings (formatted string literals) allow embedding expressions: f'{var=}'. Support format specifiers, multiline strings, expressions, and = specifier for debugging. More readable and performant than older formatting methods.
How does Python handle integer division and floating-point arithmetic?
/ performs true division (returns float), // performs floor division (returns int for ints). Floating-point follows IEEE 754. Be aware of floating-point precision issues and decimal module for exact decimal arithmetic.
What are the different ways to comment Python code?
Use # for single-line comments, triple quotes (''' or """) for multiline comments/docstrings. Docstrings document modules, classes, methods. Comments explain why, not what. Follow PEP 257 for docstring conventions.
How do you use match statements (pattern matching) in Python?
Match statements (3.10+) provide pattern matching: match value: case pattern: .... Supports patterns like literals, sequences, mappings, classes. More powerful than switch statements in other languages.
What are the naming conventions in Python?
Follow PEP 8: lowercase_with_underscores for functions/variables, CapitalizedWords for classes, _single_leading_underscore for internal use, __double_leading_underscore for name mangling, UPPERCASE for constants.
How does Python handle operator precedence?
Python follows operator precedence: exponentiation highest, then multiplication/division, then addition/subtraction. Parentheses override precedence. Bitwise operators have specific precedence. Important for complex expressions.
What are the different number systems and their representation in Python?
Support binary (0b), octal (0o), hexadecimal (0x) literals. Convert between bases using bin(), oct(), hex(), int(str, base). Important for bit manipulation and low-level operations.
What is the time complexity of dictionary operations in Python?
Average case time complexities for dictionary operations are: Get item O(1), Set item O(1), Delete item O(1), Search O(1). However, worst case can be O(n) due to hash collisions. Dictionary uses hash table implementation for efficient access.
How does a list differ from an array in Python?
Lists are dynamic arrays that can store elements of different types. Arrays (from array module) store homogeneous data, are more memory efficient, and support mathematical operations. Lists offer more flexibility but use more memory due to storing references.
Explain the implementation and use cases of heapq in Python.
heapq implements min heap, providing O(log n) push/pop operations. Used for priority queues, finding n smallest/largest elements, and scheduling tasks. Methods include heappush(), heappop(), heapify(). Can implement max heap by negating values.
What is the difference between sorted() and sort() methods?
sorted() creates new sorted list, leaving original unchanged. list.sort() modifies list in-place. Both use Timsort algorithm, accept key function and reverse parameter. sorted() works on any iterable, while sort() is list-only method.
How do you implement a binary search tree in Python?
Implement using class with Node class (value, left, right pointers) and Tree class with insert, delete, search methods. Balance tree using rotations. Time complexity O(log n) average case, O(n) worst case for unbalanced tree.
What are deques and when should they be used?
Deques (collections.deque) are double-ended queues supporting O(1) append/pop at both ends. Used for sliding windows, maintain last n items, implement queues/stacks efficiently. More efficient than lists for these operations.
How does the bisect module work in Python?
bisect module provides binary search functionality for sorted sequences. bisect_left finds insertion point for element to maintain sort order, bisect_right finds rightmost position. O(log n) time complexity. Useful for maintaining sorted sequences.
Explain how sets handle hash collisions in Python.
Sets use hash tables with open addressing to handle collisions. When collision occurs, probing finds next empty slot. Load factor determines resize threshold. Set operations (union, intersection) are optimized using hash-based algorithms.
What is the implementation of sorting algorithms in Python's list.sort()?
Python uses Timsort, hybrid of merge sort and insertion sort. Stable sort with O(n log n) worst case. Adaptive algorithm that performs well on real-world data with partially ordered sequences. Minimizes comparisons using galloping mode.
How do you implement a LinkedList in Python?
Create Node class with value and next pointer. LinkedList class manages head pointer, implements insert, delete, traverse methods. Optional: add tail pointer for O(1) append. Consider implementing iterator protocol for traversal.
What are OrderedDict and defaultdict use cases?
OrderedDict maintains insertion order (pre-3.7), useful for LRU caches. defaultdict provides default values for missing keys, simplifying handling of nested structures and counters. Both from collections module.
How do you implement graph algorithms in Python?
Represent graphs using adjacency lists (dictionaries) or matrices. Implement BFS/DFS using queue/stack. Use dict/set for visited nodes. Consider weight handling for Dijkstra/shortest paths. Implement path finding and cycle detection.
What is the difference between list and tuple for storing data?
Tuples are immutable, slightly more memory efficient, can be dictionary keys. Lists are mutable, support item assignment, have more methods. Tuples often used for returning multiple values, representing fixed collections.
How do you implement dynamic programming solutions in Python?
Use memoization (decorator or dict) or tabulation (arrays). Handle base cases, build solution using smaller subproblems. Consider space optimization using rolling arrays. Common in optimization problems.
What are the methods to handle collisions in hash tables?
Python uses open addressing with random probing. Alternative methods: chaining (linked lists), linear probing, quadratic probing. Load factor determines rehashing. Performance depends on hash function quality and collision resolution.
How do you implement a stack and queue using list?
Stack: use append() and pop() for LIFO. Queue: use append() and pop(0)/list.pop(0) for FIFO (inefficient, use collections.deque instead). Consider implementing size limits and empty checks.
What is the Counter class and how is it used?
Counter from collections creates dictionary subclass for counting hashable objects. Supports addition, subtraction, intersection, union. Methods: most_common(), elements(). Useful for frequency counting and multisets.
How do you implement a trie (prefix tree) in Python?
Create TrieNode class with children dict and is_end flag. Implement insert, search, startswith methods. Use dict for child nodes. Time complexity O(m) for operations where m is key length. Useful for autocomplete, spell check.
What are the benefits of using bytearray over bytes?
bytearray is mutable version of bytes. Supports in-place modifications, useful for building binary data incrementally. Methods like append(), extend(), reverse(). Consider for large binary data manipulation.
How do you implement binary search in Python?
Implement iteratively or recursively on sorted sequence. Handle mid calculation, comparison, and boundary updates. Time complexity O(log n). Consider bisect module for built-in implementation. Handle edge cases and duplicates.
What are the differences between list methods append() and extend()?
append() adds single element, extend() adds elements from iterable. append([1,2]) adds list as single element, extend([1,2]) adds individual elements. extend() equivalent to += operator. Consider memory and performance implications.
How do you implement a priority queue in Python?
Use heapq module for min heap implementation. Wrap elements in tuples with priority. Alternative: queue.PriorityQueue for thread-safe version. Operations: push O(log n), pop O(log n). Handle custom comparison.
What are the performance implications of string concatenation?
String concatenation with += creates new string objects. Use join() method for multiple concatenations, more efficient. For building strings, consider list of strings or io.StringIO. String interning affects memory usage.
How do you implement union-find (disjoint set) in Python?
Implement find and union operations using dictionary/array. Use path compression and union by rank optimizations. Time complexity nearly O(1) with optimizations. Used in Kruskal's algorithm, connected components.
What are the differences between range and xrange?
In Python 2, range creates list, xrange creates iterator object. Python 3's range is like xrange, memory efficient iterator. Use for loops, sequence generation. Consider memory usage for large ranges.
How do you implement memoization in Python?
Use decorator with cache dict, or functools.lru_cache. Store function arguments as keys. Handle mutable arguments. Consider cache size limits and cleanup. Used in dynamic programming, expensive computations.
What are the best practices for dictionary key selection?
Use immutable types (strings, numbers, tuples of immutables). Consider hash collisions and distribution. Custom objects need __hash__ and __eq__. Avoid mutable keys that could change hash value.
How do you implement sorting algorithms like quicksort and mergesort?
Quicksort: partition around pivot, recursively sort subarrays. Mergesort: divide array, sort recursively, merge sorted halves. Consider in-place vs new array, stability requirements, pivot selection strategies.
What are itertools functions and their use cases?
itertools provides efficient iteration tools: combinations(), permutations(), product(), cycle(), chain(). Memory efficient iterators for combinatorial operations. Used in generating test cases, processing sequences.
What are classes and objects in Python? How do they differ?
A class is a blueprint for objects, defining attributes and methods. An object is an instance of a class. Classes define structure and behavior, while objects contain actual data. Example: class Dog defines properties like breed, while a specific dog object represents an actual dog with those properties.
Explain inheritance in Python and how to implement it.
Inheritance allows a class to inherit attributes and methods from another class. Implemented using class Child(Parent). Supports single, multiple, and multilevel inheritance. Example: class Car(Vehicle) inherits from Vehicle class. Use super() to access parent class methods.
What are class methods, static methods, and instance methods? How do they differ?
Instance methods (default) have self parameter, access instance data. Class methods (@classmethod) have cls parameter, access class data. Static methods (@staticmethod) have no special first parameter, don't access class/instance data. Each serves different purpose in class design.
How does Python implement encapsulation?
Python uses name mangling with double underscore prefix (__var) for private attributes. Single underscore (_var) for protected attributes (convention). No true private variables, but follows 'we're all consenting adults' philosophy. Access control through properties and descriptors.
What are magic methods (dunder methods) and how are they used?
Magic methods control object behavior for built-in operations. Examples: __init__ for initialization, __str__ for string representation, __len__ for length, __getitem__ for indexing. Enable operator overloading and customize object behavior.
How do you implement polymorphism in Python?
Polymorphism implemented through method overriding in inheritance and duck typing. Same method name behaves differently for different classes. No need for explicit interface declarations. Example: different classes implementing same method name but different behaviors.
What are properties in Python and when should they be used?
Properties (@property decorator) provide getter/setter functionality with attribute-like syntax. Control access to attributes, add validation, make attributes read-only. Example: @property for getter, @name.setter for setter. Used for computed attributes and encapsulation.
Explain multiple inheritance and method resolution order (MRO) in Python.
Multiple inheritance allows class to inherit from multiple parents: class Child(Parent1, Parent2). MRO determines method lookup order using C3 linearization algorithm. Access MRO using Class.__mro__. Handle diamond problem through proper method resolution.
What are metaclasses and when would you use them?
Metaclasses are classes for classes, allow customizing class creation. Created using type or custom metaclass. Used for API creation, attribute/method validation, class registration, abstract base classes. Example: ABCMeta for abstract classes.
How do descriptors work in Python?
Descriptors control attribute access through __get__, __set__, __delete__ methods. Used in properties, methods, class attributes. Enable reusable attribute behavior. Example: implementing validation, computed attributes, or attribute access logging.
What is composition and how does it compare to inheritance?
Composition creates objects containing other objects as parts (has-a relationship), while inheritance creates is-a relationships. Composition more flexible, reduces coupling. Example: Car has-a Engine vs. ElectricCar is-a Car.
How do you implement abstract classes in Python?
Use abc module with ABC class and @abstractmethod decorator. Abstract classes can't be instantiated, enforce interface implementation. Example: from abc import ABC, abstractmethod. Used for defining common interfaces and ensuring implementation.
What are class and instance variables? How do they differ?
Class variables shared among all instances (defined in class), instance variables unique to each instance (defined in __init__). Class variables accessed through class or instance, modified through class. Be careful with mutable class variables.
How does super() work in Python?
super() returns proxy object for delegating method calls to parent class. Handles multiple inheritance correctly using MRO. Used in __init__ and other overridden methods. Example: super().__init__() calls parent's __init__.
What is the difference between __new__ and __init__?
__new__ creates instance, called before __init__. __init__ initializes instance after creation. __new__ rarely overridden except for singletons, immutables. __new__ is static method, __init__ is instance method.
How do you implement a singleton pattern in Python?
Implement using metaclass, __new__ method, or module-level instance. Control instance creation, ensure single instance exists. Handle thread safety if needed. Example: override __new__ to return existing instance or decorator approach.
What are slots and when should they be used?
__slots__ restricts instance attributes to fixed set, reduces memory usage. Faster attribute access, prevents dynamic attribute addition. Trade flexibility for performance. Used in classes with fixed attribute set.
How do you handle attribute access in Python classes?
Use __getattr__, __setattr__, __getattribute__, __delattr__ for custom attribute access. __getattr__ called for missing attributes, __getattribute__ for all attribute access. Be careful with infinite recursion.
What is method overriding and when should it be used?
Method overriding redefines method from parent class in child class. Used to specialize behavior while maintaining interface. Call parent method using super() when needed. Example: overriding __str__ for custom string representation.
How do you implement operator overloading in Python?
Override special methods for operators: __add__ for +, __eq__ for ==, etc. Enable class instances to work with built-in operators. Example: implement __lt__ for sorting. Consider reverse operations (__radd__, etc.).
What are class decorators and how are they used?
Class decorators modify or enhance class definitions. Applied using @decorator syntax above class. Can add attributes, methods, or modify class behavior. Example: dataclass decorator for automatic __init__, __repr__.
How do you implement immutable classes in Python?
Use __slots__, @property with only getter, override __setattr__/__delattr__. Store data in private tuples, implement __new__ instead of __init__. Consider frozen dataclasses. Make all instance data immutable.
What is the role of __repr__ vs __str__?
__str__ for human-readable string representation, __repr__ for unambiguous representation (debugging). __repr__ should be complete enough to recreate object if possible. Default to __repr__ if __str__ not defined.
How do you implement context managers using classes?
Implement __enter__ and __exit__ methods. __enter__ sets up context, __exit__ handles cleanup. Used with 'with' statement. Alternative: @contextmanager decorator for function-based approach.
What are mixins and when should they be used?
Mixins are classes providing additional functionality through multiple inheritance. Used for reusable features across different classes. Keep mixins focused, avoid state. Example: LoggerMixin for adding logging capability.
How do you handle object serialization in Python?
Use pickle for Python-specific serialization, implement __getstate__/__setstate__ for custom serialization. Consider JSON serialization with custom encoders/decoders. Handle security implications of deserialization.
What are dataclasses and when should they be used?
Dataclasses (@dataclass) automatically add generated methods (__init__, __repr__, etc.). Used for classes primarily storing data. Support comparison, frozen instances, inheritance. More concise than manual implementation.
How do you implement custom container classes?
Implement container protocol methods: __len__, __getitem__, __setitem__, __delitem__, __iter__. Optional: __contains__, __reversed__. Consider collections.abc base classes. Example: custom list or dictionary implementation.
What is the role of __dict__ in Python classes?
__dict__ stores instance attributes in dictionary. Enables dynamic attribute addition. Not present when using __slots__. Accessed for introspection, serialization. Consider memory implications for many instances.
What are lambda functions in Python and when should they be used?
Lambda functions are anonymous, single-expression functions using lambda keyword. Syntax: lambda args: expression. Best for simple operations, function arguments, and when function is used once. Example: lambda x: x*2. Limited to single expression, no statements allowed.
Explain list comprehensions and their advantages.
List comprehensions create lists using compact syntax: [expression for item in iterable if condition]. More readable and often faster than loops. Creates new list in memory. Example: [x**2 for x in range(10) if x % 2 == 0]. Good for transforming and filtering data.
How do decorators work in Python and what are their use cases?
Decorators are higher-order functions that modify other functions. Use @decorator syntax or function_name = decorator(function_name). Common uses: logging, timing, authentication, caching. Implement using nested functions or classes with __call__. Can take arguments using decorator factories.
What is the difference between map(), filter(), and reduce()?
map(func, iterable) applies function to each element. filter(func, iterable) keeps elements where function returns True. reduce(func, iterable) accumulates values using binary function. All return iterators (except reduce). Consider list comprehensions or generator expressions as alternatives.
What are generators and how do they differ from regular functions?
Generators are functions using yield keyword to return values incrementally. Create iterator objects, maintain state between calls. Memory efficient for large sequences. Used with for loops, next(). Example: def gen(): yield from range(10). Don't store all values in memory.
How do you handle variable scope and closures in Python?
Python uses LEGB scope (Local, Enclosing, Global, Built-in). Closures capture variable values from enclosing scope. Use nonlocal for enclosing scope, global for global scope. Closures common in decorators and factory functions.
What are function annotations and how are they used?
Annotations provide type hints: def func(x: int) -> str. Stored in __annotations__ dict. Not enforced at runtime, used by type checkers, IDEs, documentation. PEP 484 defines type hinting standards. Help with code understanding and verification.
How do you implement partial functions in Python?
Use functools.partial to create new function with fixed arguments. Reduces function arity by pre-filling arguments. Example: from functools import partial; new_func = partial(original_func, arg1). Useful for callback functions and function factories.
What are the different types of function arguments in Python?
Positional arguments (standard), keyword arguments (name=value), default arguments (def func(x=1)), variable arguments (*args for tuple, **kwargs for dict). Order matters: positional, *args, keyword, **kwargs. Consider argument flexibility vs clarity.
How do you implement function caching/memoization?
Use @functools.lru_cache decorator or implement custom caching with dict. Stores function results, returns cached value for same arguments. Consider cache size, argument hashability. Example: @lru_cache(maxsize=None). Good for expensive computations.
What are generator expressions and when should they be used?
Generator expressions create iterator objects: (expression for item in iterable). Like list comprehensions but more memory efficient. Use when iterating once over large sequences. Syntax uses parentheses: sum(x*2 for x in range(1000)).
How do you implement currying in Python?
Currying transforms function with multiple arguments into series of single-argument functions. Implement using nested functions or partial application. Example: def curry(f): return lambda x: lambda y: f(x,y). Used for function composition and partial application.
What is recursion and what are its limitations in Python?
Recursion is function calling itself. Limited by recursion depth (default 1000). Use tail recursion optimization or iteration for deep recursion. Example: factorial implementation. Consider stack overflow risk and performance implications.
How do you work with higher-order functions?
Higher-order functions take functions as arguments or return functions. Examples: map, filter, decorators. Enable function composition and abstraction. Consider type hints for function arguments. Common in functional programming patterns.
What are pure functions and why are they important?
Pure functions always return same output for same input, have no side effects. Benefits: easier testing, debugging, parallelization. Example: def add(x,y): return x+y. Avoid global state, I/O operations. Key concept in functional programming.
How do you handle function documentation in Python?
Use docstrings (triple quotes) for function documentation. Include description, parameters, return value, examples. Access via help() or __doc__. Follow PEP 257 conventions. Consider tools like Sphinx for documentation generation.
What are the benefits of using dict/set comprehensions?
Dict comprehension: {key:value for item in iterable}. Set comprehension: {expression for item in iterable}. More concise than loops, creates new dict/set. Example: {x:x**2 for x in range(5)}. Consider readability vs complexity.
How do you implement function overloading in Python?
Python doesn't support traditional overloading. Use default arguments, *args, **kwargs, or @singledispatch decorator. Alternative: multiple dispatch based on argument types. Consider interface clarity vs flexibility.
What are coroutines and how do they differ from generators?
Coroutines use async/await syntax, support concurrent programming. Unlike generators, can receive values with send(). Used with asyncio for asynchronous operations. Example: async def coro(): await operation(). Consider event loop integration.
How do you handle nested functions and variable scope?
Nested functions have access to outer function variables (closure). Use nonlocal keyword to modify enclosed variables. Common in decorators and callbacks. Consider readability and maintenance implications.
What is function composition and how is it implemented?
Function composition combines functions: f(g(x)). Implement using higher-order functions or operator module. Example: compose = lambda f, g: lambda x: f(g(x)). Consider readability and error handling.
How do you implement iterator protocol in custom objects?
Implement __iter__ and __next__ methods. __iter__ returns iterator object, __next__ provides next value or raises StopIteration. Used in for loops, generators. Example: custom range implementation. Consider memory efficiency.
What are the benefits of using functools module?
functools provides tools for functional programming: reduce, partial, lru_cache, wraps, singledispatch. Enhances function manipulation and optimization. Example: @wraps preserves function metadata in decorators.
How do you handle function attributes and metadata?
Functions are objects with attributes. Access/set via function.__dict__. Common attributes: __name__, __doc__, __module__. Use @wraps to preserve metadata in decorators. Consider documentation and debugging implications.
What is tail recursion optimization?
Tail recursion occurs when recursive call is last operation. Python doesn't optimize tail recursion automatically. Convert to iteration or use trampolining for optimization. Consider stack overflow prevention.
How do you implement method chaining?
Return self from methods to enable chaining: obj.method1().method2(). Common in builder pattern and fluent interfaces. Consider readability and maintenance. Example: query builders, object configuration.
What are the patterns for handling optional function parameters?
Use default arguments, None values, or *args/**kwargs. Consider parameter order, default mutability issues. Example: def func(required, optional=None). Document parameter optionality clearly.
How do you implement function factories?
Function factories create and return new functions. Use closures to capture configuration. Common in parameterized decorators, specialized functions. Example: def make_multiplier(n): return lambda x: x * n.
What are the best practices for error handling in functional programming?
Use Either/Maybe patterns, return tuple of (success, result), or raise exceptions. Handle errors explicitly in function composition. Consider monadic error handling. Maintain functional purity where possible.
What is the basic structure of exception handling in Python?
Basic structure uses try/except blocks: try to execute code that might raise exception, except to handle specific exceptions. Optional else clause for code when no exception occurs, finally for cleanup. Example: try: risky_operation() except TypeError: handle_error()
How do you create and raise custom exceptions?
Create custom exceptions by inheriting from Exception class or specific exception types. Raise using raise keyword. Include meaningful error messages and attributes. Example: class CustomError(Exception): pass; raise CustomError('message')
What is the purpose of the 'finally' clause in exception handling?
finally clause executes regardless of whether exception occurred or was handled. Used for cleanup operations like closing files, database connections. Executes even if return, break, or continue occurs in try/except. Ensures resource cleanup.
How do you implement logging in Python applications?
Use logging module with different levels (DEBUG, INFO, WARNING, ERROR, CRITICAL). Configure handlers, formatters, log destinations. Example: logging.basicConfig(level=logging.INFO). Consider log rotation, timestamp formats, context information.
What is the difference between pdb and ipdb debuggers?
pdb is Python's built-in debugger, ipdb adds IPython features like tab completion, syntax highlighting. Both support stepping through code, inspecting variables, setting breakpoints. Commands: n(next), s(step), c(continue), q(quit).
How do you handle multiple exceptions in a single except clause?
Group exceptions in tuple: except (TypeError, ValueError) as e. Access exception info using 'as' keyword. Consider exception hierarchy, order from specific to general. Handle each exception type appropriately.
What are context managers and how do they help in error handling?
Context managers (with statement) ensure proper resource cleanup. Implement __enter__ and __exit__ methods or use contextlib.contextmanager. Handle exceptions in __exit__. Example: file handling, database connections.
How do you debug memory leaks in Python?
Use memory_profiler, tracemalloc, gc module. Monitor object references, circular references. Tools: objgraph for visualization, sys.getsizeof() for object size. Consider weak references, proper cleanup in __del__.
What is the difference between assertions and exceptions?
Assertions (assert condition, message) check program correctness, disabled with -O flag. Exceptions handle runtime errors. Assertions for debugging/development, exceptions for runtime error handling. Use assertions for invariants.
How do you implement error tracking in production applications?
Use error tracking services (Sentry, Rollbar). Implement custom error handlers, logging middleware. Capture stack traces, context information. Consider environment-specific handling, error aggregation, alert thresholds.
What are decorators for debugging and error handling?
Create decorators for timing, logging, error catching. Example: @log_errors, @retry, @timeout. Handle function metadata using functools.wraps. Consider performance impact, logging levels.
How do you handle exceptions in multithreaded applications?
Use thread-specific exception handlers, threading.excepthook. Handle exceptions within thread function. Consider thread safety in logging, global error handlers. Implement proper thread cleanup.
What are the best practices for exception hierarchy design?
Create base exception class for application. Structure hierarchy based on error types. Include relevant error information, maintain backwards compatibility. Example: AppError -> ValidationError -> FieldValidationError.
How do you debug infinite loops and recursion issues?
Use breakpoints, print debugging, sys.setrecursionlimit(). Monitor stack traces, memory usage. Tools: pdb for stepping through code. Implement timeout mechanisms, recursion depth checking.
What is the role of sys.excepthook in error handling?
sys.excepthook handles uncaught exceptions. Customize for global error handling, logging. Access exception type, value, traceback. Consider different handling for different environments (dev/prod).
How do you implement retry logic for operations?
Use decorators or context managers for retries. Handle specific exceptions, implement backoff strategy. Example: @retry(times=3, exceptions=(NetworkError,)). Consider timeout, max attempts, delay between retries.
What debugging tools are available in Python IDEs?
IDEs provide integrated debuggers, breakpoints, variable inspection, call stack viewing. Popular: PyCharm debugger, VS Code debugger. Features: conditional breakpoints, expression evaluation, watch variables.
How do you handle exceptions in async code?
Use try/except in async functions, handle event loop exceptions. asyncio.excepthook for loop exceptions. Consider concurrent error handling, task cancellation. Example: async with AsyncExitStack().
What are the patterns for error reporting in API development?
Implement consistent error response format, HTTP status codes. Include error codes, messages, debug information. Consider API versioning, client error handling. Example: {error: {code: 'VAL_001', message: 'Invalid input'}}.
How do you debug performance issues in Python code?
Use profilers (cProfile, line_profiler), timing decorators. Monitor CPU usage, memory allocation. Tools: py-spy for sampling profiler. Consider bottlenecks, optimization opportunities.
What are the best practices for exception messages?
Include specific error details, context information. Make messages user-friendly, actionable. Consider internationalization, security implications. Example: 'Failed to connect to database at host:port: timeout after 30s'.
How do you implement error boundaries in Python applications?
Create wrapper classes/functions for error containment. Handle subsystem failures gracefully. Consider component isolation, fallback behavior. Example: class ErrorBoundary context manager.
What are the techniques for debugging third-party library issues?
Use source-level debugging, monkey patching. Inspect library source, logging integration. Consider version compatibility, issue tracking. Tools: debugger step-into, logging interception.
How do you handle database-related errors effectively?
Implement specific exception handling for DB errors. Handle connection issues, transaction rollback. Consider retry logic, connection pooling. Example: handle SQLAlchemy exceptions specifically.
What are the strategies for debugging concurrent code?
Use thread/process-specific logging, race condition detection. Tools: threading.VERBOSE, multiprocessing debug logs. Consider deadlock detection, thread sanitizers.
How do you implement error recovery mechanisms?
Create recovery procedures for critical errors. Implement state restoration, data consistency checks. Consider transaction boundaries, cleanup operations. Example: backup/restore mechanisms.
What are the patterns for handling configuration errors?
Validate configuration early, provide clear error messages. Handle missing/invalid settings gracefully. Consider environment-specific defaults, configuration versioning. Example: config validation decorators.
How do you debug memory fragmentation issues?
Use memory_profiler, guppy3 for heap analysis. Monitor object lifecycle, memory patterns. Consider garbage collection timing, object pooling. Tools: objgraph for reference visualization.
What are the best practices for logging sensitive information?
Implement data masking, logging filters. Handle PII, credentials securely. Consider compliance requirements, log retention policies. Example: custom log formatters for sensitive data.
What are the different modes for opening files in Python?
Common modes: 'r' (read), 'w' (write), 'a' (append), 'b' (binary), '+' (read/write). Can combine modes: 'rb' (read binary), 'w+' (read/write). Default is text mode, 'b' for binary. Consider encoding parameter for text files.
How do you use context managers with files?
Use 'with' statement: 'with open(filename) as f:'. Ensures proper file closure even if exceptions occur. Best practice over manual open/close. Can handle multiple files: 'with open(f1) as a, open(f2) as b:'.
What is the difference between read(), readline(), and readlines()?
read() reads entire file into string, readline() reads single line, readlines() reads all lines into list. read() can specify size in bytes. Consider memory usage for large files. Use iteration for efficient line reading.
How do you handle CSV files in Python?
Use csv module: reader, writer, DictReader, DictWriter classes. Handle delimiters, quoting, escaping. Consider newline='' parameter when opening. Example: csv.DictReader for named columns. Handle different dialects.
What are the best practices for working with JSON files?
Use json module: json.load(file), json.dump(data, file). Handle encoding, pretty printing (indent parameter). Consider custom serialization (default parameter). Handle JSON parsing errors, large files.
How do you handle binary file operations?
Use 'b' mode flag, read/write bytes objects. Methods: read(size), write(bytes). Consider bytearray for mutable binary data. Handle endianness, struct module for binary structures. Buffer protocols.
What are file buffering modes in Python?
Buffering options: 0 (unbuffered), 1 (line buffered), >1 (size in bytes). Default is system dependent. Set with buffering parameter. Consider performance implications, memory usage. Flush buffer manually with flush().
How do you handle file encodings?
Specify encoding parameter in open(): UTF-8, ASCII, etc. Handle encoding/decoding errors with errors parameter. Use codecs module for advanced encoding. Consider BOM, default system encoding.
What is memory mapping and when should it be used?
Use mmap module for memory-mapped file I/O. Treats file as memory buffer. Good for large files, shared memory. Consider platform differences, file size limitations. Handle proper cleanup.
How do you implement file locking?
Use fcntl module (Unix) or msvcrt (Windows). Implement advisory locking. Handle shared vs exclusive locks. Consider timeout, deadlock prevention. Example: with FileLock(filename):.
What are temporary files and how to use them?
Use tempfile module: NamedTemporaryFile, TemporaryDirectory. Auto-cleanup when closed. Secure creation, unique names. Consider cleanup on program exit. Handle permissions properly.
How do you handle large file processing efficiently?
Use generators for line iteration, chunk reading. Consider memory usage, buffering. Use mmap for random access. Implement progress tracking. Handle cleanup properly. Consider parallel processing.
What are file-like objects and when to use them?
Objects implementing file interface (read, write, etc.). Examples: StringIO, BytesIO for in-memory files. Used for compatibility with file operations. Consider context manager implementation.
How do you handle file system operations safely?
Use os.path or pathlib for path operations. Handle permissions, existence checks. Consider race conditions, atomic operations. Implement proper error handling. Use shutil for high-level operations.
What are the patterns for handling configuration files?
Use configparser for INI files, yaml for YAML. Handle defaults, validation. Consider environment overrides. Implement config reloading. Handle sensitive data properly.
How do you implement file watching/monitoring?
Use watchdog library or platform-specific APIs. Handle file system events. Consider polling vs event-based. Implement proper cleanup. Handle recursive monitoring.
What are the best practices for file path handling?
Use pathlib.Path for object-oriented path operations. Handle platform differences, relative paths. Consider path normalization, validation. Handle special characters, spaces.
How do you handle Excel files in Python?
Use openpyxl, pandas for XLSX files. xlrd/xlwt for older formats. Handle sheets, formatting, formulas. Consider memory usage for large files. Implement proper cleanup.
What are the techniques for file compression handling?
Use gzip, zipfile, tarfile modules. Handle compression levels, passwords. Consider streaming for large files. Implement progress tracking. Handle multiple file archives.
How do you implement file searching and pattern matching?
Use glob, fnmatch for patterns. re module for regex. Consider recursive search, filters. Handle large directories efficiently. Implement proper error handling.
What are the patterns for handling log files?
Use rotating file handlers, proper formatting. Handle log levels, rotation size. Consider compression, retention policy. Implement proper cleanup. Handle concurrent access.
How do you handle file metadata operations?
Use os.stat, Path.stat() for metadata. Handle timestamps, permissions. Consider platform differences. Implement proper error handling. Handle symbolic links.
What are atomic file operations and how to implement them?
Use os.replace for atomic writes. Implement write-to-temp-then-rename pattern. Handle concurrent access. Consider file system limitations. Implement proper error recovery.
How do you handle file system permissions securely?
Use os.chmod, Path.chmod() for permissions. Handle umask settings. Consider security implications. Implement least privilege principle. Handle permission inheritance.
What are the strategies for file backup and versioning?
Implement backup rotation, version naming. Handle incremental backups. Consider compression, deduplication. Implement proper cleanup. Handle backup verification.
How do you handle network file operations?
Use appropriate protocols (FTP, SFTP). Handle timeouts, retries. Consider security, authentication. Implement progress tracking. Handle network errors properly.
What are the best practices for file cleanup?
Use context managers, atexit handlers. Implement proper error handling. Consider temporary files, locks. Handle program crashes. Implement cleanup verification.
How do you implement file type detection?
Use mimetypes module, file signatures. Handle binary vs text files. Consider encoding detection. Implement proper validation. Handle unknown file types.
What are the techniques for file merging and splitting?
Handle chunk size, ordering. Implement progress tracking. Consider memory efficiency. Handle partial failures. Implement verification steps.
What are different ways to import modules in Python?
Common import methods: 'import module', 'from module import item', 'from module import *', 'import module as alias'. Each has different namespace effects. 'import *' not recommended due to namespace pollution. Consider relative vs absolute imports.
What is __init__.py and what is its purpose?
__init__.py marks directory as Python package, can initialize package attributes, execute package initialization code. Can be empty. Controls what's exported with __all__. Essential for Python 2, optional but recommended for Python 3.
How does Python's module search path work?
Python searches modules in order: current directory, PYTHONPATH, standard library directories, site-packages. Accessible via sys.path. Can modify runtime using sys.path.append(). Consider virtual environment impact on search path.
What are relative imports and when should they be used?
Relative imports use dots: from . import module (current package), from .. import module (parent package). Only work within packages. Use explicit relative imports for clarity. Consider package restructuring implications.
How do you create and use virtual environments?
Use venv module: python -m venv env_name. Activate using source env/bin/activate (Unix) or env\Scripts\activate (Windows). Manages project-specific dependencies. Consider requirements.txt for dependency tracking.
What is the purpose of setup.py and how is it used?
setup.py defines package metadata, dependencies, entry points for distribution. Used with setuptools for package building and installation. Includes version, requirements, package data. Consider modern alternatives like pyproject.toml.
How do namespace packages work in Python?
Namespace packages allow splitting package across multiple directories. No __init__.py required. Uses implicit namespace package mechanism. Useful for plugin systems, large frameworks. Consider version compatibility issues.
What is the difference between pip and conda?
pip is Python's package installer, conda is cross-platform package/environment manager. pip works with PyPI, conda has own repositories. conda handles non-Python dependencies. Consider project requirements for choosing.
How do you handle circular imports?
Avoid using import statements inside functions, use dependency injection, restructure code to break cycles. Consider lazy imports, import inside functions. Signals possible design issues. Review module dependencies.
What is __all__ and how is it used?
__all__ list controls what's imported with 'from module import *'. Explicitly defines public API. Good practice for large modules. Example: __all__ = ['func1', 'func2']. Consider documentation and maintenance.
How do you distribute Python packages?
Build distribution using setuptools/wheel, upload to PyPI using twine. Create source and wheel distributions. Consider versioning, documentation, licenses. Handle package data, dependencies correctly.
What are entry points and how are they used?
Entry points define console scripts, plugin points in setup.py/pyproject.toml. Enable command-line tools, plugin systems. Format: name=module:function. Consider cross-platform compatibility, documentation.
How do you manage project dependencies effectively?
Use requirements.txt or pyproject.toml, specify version ranges appropriately. Consider dev vs production dependencies. Use pip-tools or poetry for dependency management. Handle transitive dependencies.
What is the purpose of __name__ == '__main__'?
Checks if module is run directly or imported. Common for script entry points, testing. Code under this block only runs when module executed directly. Consider module reusability, testing implications.
How do you handle package data files?
Use MANIFEST.in, package_data in setup.py, include_package_data=True. Access using pkg_resources or importlib.resources. Consider data file locations, installation requirements. Handle path resolution.
What are wheel files and their advantages?
Wheels are built package format, faster installation than source distributions. Contains pre-built files, metadata. Reduces installation time, ensures consistency. Consider pure Python vs platform-specific wheels.
How do you implement plugin systems using packages?
Use entry points, namespace packages, or import hooks. Define plugin interface, discovery mechanism. Consider version compatibility, security. Handle plugin loading/unloading, configuration.
What is importlib and how is it used?
importlib provides import mechanism implementation, custom importers. Used for dynamic imports, import hooks. Access to import internals. Consider performance, security implications. Handle import errors.
How do you handle version conflicts between packages?
Use virtual environments, specify version constraints carefully. Consider dependency resolution tools, container isolation. Handle conflicts through requirement specifications. Review dependency tree.
What is pyproject.toml and its benefits?
Modern configuration file for Python projects (PEP 518). Specifies build system requirements, project metadata. Replaces setup.py, setup.cfg. Used by poetry, flit. Consider migration from setuptools.
How do you structure large Python applications?
Use packages for logical grouping, maintain clear hierarchy. Consider separation of concerns, circular dependencies. Implement proper initialization. Handle configuration, plugin systems appropriately.
What are the best practices for module documentation?
Use docstrings, maintain README files, generate API documentation. Follow PEP 257. Include usage examples, installation instructions. Consider documentation generation tools (Sphinx). Maintain changelog.
How do you implement lazy loading of modules?
Import modules when needed, use importlib.import_module(). Consider performance implications, circular dependencies. Handle import errors appropriately. Implement proper cleanup.
What are the patterns for package initialization?
Use __init__.py for package setup, expose public API. Handle optional dependencies, perform checks. Consider backwards compatibility. Implement proper error handling.
How do you handle conditional imports?
Use try/except for optional imports, implement fallbacks. Consider platform-specific modules. Handle missing dependencies gracefully. Document requirements clearly.
What are import hooks and when to use them?
Import hooks customize module importing process. Implement custom importing behavior, transformations. Used for special loading requirements. Consider performance impact, maintenance complexity.
How do you manage package versions effectively?
Use semantic versioning, maintain changelog. Consider backwards compatibility, deprecation policy. Handle version bumping, release process. Document version requirements clearly.
What are the security considerations for package management?
Verify package sources, use package hashes. Consider supply chain attacks, dependency scanning. Implement security updates policy. Review dependencies regularly.
How do you implement cross-platform package compatibility?
Handle platform-specific code, use conditional imports. Consider environment differences. Test on multiple platforms. Handle path separators, line endings. Document platform requirements.
What are the main differences between unittest and pytest frameworks?
unittest is built-in, class-based, requires test classes inheriting from TestCase. pytest is more flexible, supports function-based tests, better fixtures, parametrization, and plugins. pytest has more powerful assertions and better error reporting. Consider project needs for framework choice.
How do fixtures work in pytest and what are their benefits?
Fixtures provide reusable test setup/teardown, defined using @pytest.fixture decorator. Support dependency injection, different scopes (function, class, module, session). Enable clean test organization, resource sharing. Example: database connections, test data setup.
What is mocking and how is it implemented in Python tests?
Mocking replaces real objects with test doubles. Use unittest.mock or pytest-mock. MagicMock/Mock classes provide automatic attribute creation. Common uses: external services, databases, file operations. Consider patch decorator/context manager.
How do you measure and improve test coverage?
Use coverage.py or pytest-cov. Run tests with coverage collection, generate reports. Analyze uncovered lines, branches. Set minimum coverage requirements. Consider meaningful vs. superficial coverage. Focus on critical paths.
What is Test-Driven Development (TDD) and how is it practiced?
TDD cycle: write failing test, write code to pass, refactor. Tests drive design, document requirements. Write minimal code to pass tests. Benefits: better design, regression protection, documentation. Consider Red-Green-Refactor cycle.
How do you implement parameterized testing in pytest?
Use @pytest.mark.parametrize decorator to run same test with different inputs. Supports multiple parameters, custom IDs. Reduces test code duplication. Example: @pytest.mark.parametrize('input,expected', [(1,2), (2,4)]). Consider data organization.
What are pytest markers and how are they used?
Markers (@pytest.mark) categorize tests, control execution. Built-in markers: skip, skipif, xfail. Custom markers for test organization, selection. Register markers in pytest.ini. Consider marker documentation, organization.
How do you handle database testing?
Use test databases, fixtures for setup/teardown. Consider transaction rollback, database isolation. Mock database when appropriate. Implement proper cleanup. Use tools like pytest-django for framework-specific support.
What is the purpose of test doubles (mocks, stubs, fakes)?
Test doubles replace real dependencies. Mocks verify interactions, stubs provide canned responses, fakes implement lightweight alternatives. Choose based on test needs. Consider interaction vs. state testing.
How do you test async code in Python?
Use pytest-asyncio for async tests. Mark tests with @pytest.mark.asyncio. Handle coroutines properly. Consider event loop management. Test async contexts, timeouts. Handle async cleanup properly.
What are best practices for test organization?
Group related tests, use clear naming conventions. Separate unit/integration tests. Follow AAA pattern (Arrange-Act-Assert). Maintain test independence. Consider test discoverability, maintenance.
How do you implement integration testing?
Test component interactions, external services. Use appropriate fixtures, mocking selectively. Consider test environment setup. Handle cleanup properly. Balance coverage vs. execution time.
What is property-based testing and how is it implemented?
Use hypothesis library for property-based testing. Define properties, let framework generate test cases. Useful for finding edge cases. Consider strategy definition, test case generation. Handle test case reduction.
How do you handle test data management?
Use fixtures, factory libraries (factory_boy). Consider data isolation, cleanup. Implement proper test data generation. Handle complex data relationships. Consider data versioning, maintenance.
What are pytest conftest.py files and their purpose?
conftest.py provides shared fixtures across multiple test files. Defines test configuration, custom markers. Supports fixture overriding, plugin hooks. Consider scope organization, reusability.
How do you implement performance testing?
Use pytest-benchmark for performance tests. Measure execution time, resource usage. Consider baseline comparisons, statistical analysis. Handle environment variations. Document performance requirements.
What is monkey patching and when should it be used?
Monkey patching modifies objects/modules at runtime for testing. Use pytest.monkeypatch fixture. Handle cleanup properly. Consider implications on test isolation. Use sparingly, prefer dependency injection.
How do you test exception handling?
Use pytest.raises context manager or unittest.assertRaises. Test exception types, messages. Consider exception inheritance, multiple exceptions. Test cleanup handling. Verify exception context.
What are pytest plugins and how are they used?
Plugins extend pytest functionality. Common plugins: pytest-cov, pytest-mock, pytest-django. Install via pip, configure in pytest.ini. Consider plugin interactions, maintenance. Document plugin requirements.
How do you implement API testing?
Use requests, pytest-httpx for HTTP testing. Mock external services appropriately. Consider response validation, error cases. Handle authentication, rate limiting. Test different HTTP methods.
What is behavior-driven development (BDD) in Python?
Use pytest-bdd or behave for BDD. Write tests in Gherkin syntax. Map steps to test code. Consider stakeholder communication. Balance readability vs. maintenance. Document behavior specifications.
How do you handle environment-specific testing?
Use environment variables, configuration files. Implement test environment management. Consider CI/CD integration. Handle sensitive data properly. Document environment requirements.
What are testing anti-patterns to avoid?
Avoid test interdependence, slow tests, excessive mocking. Don't test implementation details. Avoid non-deterministic tests. Consider maintenance cost. Document test assumptions clearly.
How do you implement concurrent test execution?
Use pytest-xdist for parallel testing. Consider test isolation, shared resources. Handle race conditions. Implement proper cleanup. Balance parallelism vs. resource usage.
What are the strategies for testing logging?
Use caplog fixture in pytest. Verify log messages, levels. Consider log handlers, formatting. Test logger configuration. Handle temporary logger modifications.
How do you implement security testing?
Test input validation, authentication, authorization. Use security testing tools (bandit). Consider vulnerability scanning. Test security configurations. Document security requirements.
What are fixtures scope levels and when to use each?
Scopes: function (default), class, module, session. Choose based on resource costs, test isolation needs. Consider cleanup timing. Handle dependencies between fixtures. Document scope requirements.
How do you implement continuous testing?
Integrate tests in CI/CD pipeline. Automate test execution, reporting. Consider test selection, prioritization. Handle test failures appropriately. Document test requirements.
What are the patterns for testing GUI applications?
Use PyTest-Qt, PyAutoGUI for GUI testing. Handle event loops properly. Consider screenshot comparisons. Test user interactions. Handle window management. Document visual requirements.
What are the key functions in the collections module and their use cases?
Collections module provides specialized container types: defaultdict (automatic default values), Counter (counting hashable objects), deque (double-ended queue), namedtuple (tuple with named fields), OrderedDict (ordered dictionary pre-3.7). Each optimized for specific use cases and performance requirements.
How do you use the datetime module for time zone aware operations?
datetime module handles dates and times. Use datetime.datetime with tzinfo for timezone awareness, pytz for reliable timezone handling. Methods: astimezone(), replace(tzinfo=), timezone conversions. Consider DST transitions, UTC conversions.
What are the main features of the itertools module?
itertools provides functions for efficient iteration: combinations(), permutations(), product() for combinatorics; cycle(), repeat() for infinite iterators; chain(), islice() for iterator manipulation. Memory efficient for large datasets.
How do you use the os module for system operations?
os module provides OS-independent interface for operating system operations. Functions: os.path for path manipulation, os.environ for environment variables, os.walk for directory traversal. Consider platform differences, security implications.
What features does the json module provide for data serialization?
json module handles JSON encoding/decoding. Methods: dumps()/loads() for strings, dump()/load() for files. Supports custom serialization with default/object_hook. Handle encoding issues, pretty printing, security considerations.
How do you use re module for regular expressions?
re module provides regular expression operations: compile(), match(), search(), findall(). Supports patterns, groups, flags. Consider using raw strings (r'pattern'), pre-compilation for performance. Handle different regex patterns and flags.
What are the key features of the concurrent.futures module?
concurrent.futures provides high-level interface for asynchronous execution: ThreadPoolExecutor for I/O-bound tasks, ProcessPoolExecutor for CPU-bound tasks. Supports map(), submit(), as_completed(). Handle thread/process management, exceptions.
How do you use the logging module effectively?
logging provides flexible event logging: different levels (DEBUG to CRITICAL), handlers, formatters. Configure using basicConfig() or dictConfig(). Consider log rotation, formatting, handling in different environments.
What functionality does the pathlib module offer?
pathlib provides object-oriented interface for file system paths. Path class methods: glob(), mkdir(), touch(), resolve(). More readable than os.path, handles platform differences. Consider migration from os.path.
How do you use the functools module for function manipulation?
functools provides tools for functional programming: partial() for partial application, lru_cache() for caching, reduce() for reduction operations. Includes wraps() for preserving function metadata. Consider performance implications.
What features does the argparse module provide?
argparse handles command-line argument parsing: argument types, help messages, subcommands. Supports required/optional arguments, default values, custom actions. Consider user interface design, error handling.
How do you use the sqlite3 module for database operations?
sqlite3 provides SQLite database interface: connection management, cursor operations, parameter substitution. Supports transactions, custom row factories. Consider connection lifecycle, error handling, SQL injection prevention.
What are the key features of the asyncio module?
asyncio provides infrastructure for async/await code: event loops, coroutines, tasks. Supports async I/O operations, concurrency. Handle task scheduling, synchronization primitives, error handling.
How do you use the contextlib module?
contextlib provides utilities for context managers: @contextmanager decorator, ExitStack for multiple contexts. Supports resource management, cleanup operations. Consider error handling, nested contexts.
What functionality does the csv module provide?
csv handles CSV file operations: reading, writing, different dialects. Supports DictReader/DictWriter for named columns. Handle different formats, encoding issues, custom dialects. Consider large file handling.
How do you use the threading module?
threading provides high-level threading interface: Thread class, synchronization primitives (Lock, Event). Handle thread creation, synchronization, communication. Consider thread safety, deadlock prevention.
What features does the random module offer?
random provides pseudo-random number generation: randint(), choice(), shuffle(). Supports different distributions, seeding. Consider cryptographic needs (use secrets instead), reproducibility requirements.
How do you use the pickle module for serialization?
pickle handles Python object serialization: dump()/load() for files, dumps()/loads() for bytes. Consider security implications, version compatibility. Handle custom object serialization, protocol versions.
What are the key features of the sys module?
sys provides system-specific parameters/functions: sys.path for module search, sys.argv for command arguments, sys.stdin/stdout/stderr for I/O. Handle interpreter interaction, system limitations.
How do you use the shutil module?
shutil provides high-level file operations: copyfile(), rmtree(), make_archive(). Supports file copying, removal, archiving. Handle permissions, recursive operations, platform differences.
What functionality does the urllib module provide?
urllib handles URL operations: request handling, parsing, encoding. Modules: request for HTTP, parse for URL parsing, error for exceptions. Consider error handling, security, timeout configuration.
How do you use the time module effectively?
time provides time-related functions: time() for timestamps, sleep() for delays, strftime() for formatting. Handle different time representations, platform differences. Consider timezone implications.
What features does the enum module offer?
enum provides enumeration support: Enum class, auto() for automatic values. Supports unique/non-unique values, custom behavior. Consider type safety, value comparison, serialization needs.
How do you use the subprocess module?
subprocess manages external processes: run(), Popen for process control. Handle command execution, I/O redirection, process communication. Consider security, platform differences, error handling.
What are the key features of the statistics module?
statistics provides statistical functions: mean(), median(), mode(), stdev(). Supports different types of averages, variance calculations. Consider numerical stability, data types, population vs sample.
How do you use the multiprocessing module?
multiprocessing provides process-based parallelism: Process class, Pool for worker pools. Handle process creation, communication, synchronization. Consider GIL bypass, resource sharing, cleanup.
What functionality does the tempfile module provide?
tempfile handles temporary files/directories: TemporaryFile, NamedTemporaryFile, TemporaryDirectory. Supports secure creation, automatic cleanup. Consider platform differences, cleanup guarantees.
How do you use the operator module?
operator provides function equivalents of operators: itemgetter(), attrgetter() for data access, add(), mul() for arithmetic. Useful in functional programming, sorting. Consider performance implications.
What features does the platform module offer?
platform provides system information: system(), machine(), python_version(). Access hardware, OS details, Python environment. Consider cross-platform compatibility, information security.
What are the main differences between SQLAlchemy Core and ORM?
SQLAlchemy Core provides SQL abstraction layer, direct table operations. ORM provides object-relational mapping, domain model abstraction. Core offers better performance, more control. ORM provides higher-level abstractions, easier object manipulation. Consider use case requirements for choosing.
How do you handle database connection pooling in Python?
Use connection pools (SQLAlchemy's Pool, psycopg2's pool). Configure pool size, overflow, timeout. Handle connection recycling, cleanup. Consider concurrent access patterns, resource limits. Implement proper error handling and monitoring.
What are the best practices for handling database migrations?
Use migration tools (Alembic, Django migrations). Version control migrations, test before deployment. Handle data migrations separately. Consider rollback strategies, large table modifications. Document migration steps and dependencies.
How do you implement database transactions in Python?
Use transaction context managers, explicit commit/rollback. Handle atomic operations, savepoints. Implement proper error handling. Consider isolation levels, deadlock prevention. Manage transaction scope and nesting.
What are the strategies for handling database query optimization?
Use indexing, query analysis tools. Optimize JOIN operations, limit result sets. Consider query execution plans. Implement caching strategies. Monitor query performance. Use database-specific optimization features.
How do you prevent SQL injection in Python applications?
Use parameterized queries, ORM query builders. Never concatenate SQL strings. Validate input data. Consider escape sequences, prepared statements. Implement proper access controls and input sanitization.
What are the patterns for handling database relationships in ORMs?
Define relationships (one-to-many, many-to-many) using relationship() in SQLAlchemy or ForeignKey in Django. Handle lazy loading, eager loading. Consider cascade operations, backref relationships. Manage relationship lifecycle.
How do you handle database schema versioning?
Use migration tools with version control. Implement forward/backward migrations. Track schema changes in source control. Consider database branching strategies. Handle schema conflicts and dependencies.
What are the best practices for database error handling?
Handle specific database exceptions, implement retry logic. Log errors appropriately. Consider transaction rollback, connection recovery. Implement proper cleanup. Provide meaningful error messages.
How do you implement database sharding in Python?
Use sharding keys, implement routing logic. Handle cross-shard queries, transactions. Consider data distribution, rebalancing. Implement proper shard management. Handle shard failures and recovery.
What are the strategies for database backup and recovery?
Implement regular backups, verify backup integrity. Handle incremental backups, point-in-time recovery. Consider backup automation, retention policies. Test recovery procedures. Document backup/restore processes.
How do you handle database connection retries and timeouts?
Implement exponential backoff, max retry attempts. Handle connection timeouts, dead connections. Consider circuit breaker pattern. Implement proper logging and monitoring. Handle cleanup properly.
What are the patterns for handling database caching?
Use caching layers (Redis, Memcached). Implement cache invalidation strategies. Handle cache consistency. Consider cache warming, eviction policies. Implement proper error handling for cache failures.
How do you implement database replication in Python?
Configure master-slave replication, handle failover. Implement read/write splitting. Consider replication lag, consistency requirements. Handle replication errors and recovery. Monitor replication status.
What are the approaches for handling large-scale data migrations?
Implement batched operations, progress tracking. Handle data validation, rollback capability. Consider performance impact, downtime requirements. Test migration procedures. Document migration steps.
How do you implement database monitoring and logging?
Monitor query performance, connection usage. Log slow queries, errors. Implement performance metrics collection. Consider monitoring tools integration. Handle alert threshold configuration.
What are the patterns for handling database concurrency?
Use proper isolation levels, row-level locking. Handle deadlock detection, prevention. Consider optimistic vs pessimistic locking. Implement proper transaction boundaries. Handle concurrent access patterns.
How do you handle database schema changes in production?
Implement zero-downtime migrations. Handle backward compatibility. Use temporary tables for large changes. Consider rollback procedures. Test migration scripts. Document change procedures.
What are the strategies for database testing?
Use test databases, fixtures. Implement transaction rollback for tests. Handle database isolation. Consider performance testing. Test migration scripts. Implement proper cleanup procedures.
How do you handle database security and access control?
Implement proper authentication, authorization. Use least privilege principle. Handle sensitive data encryption. Consider audit logging. Implement access control policies. Handle security patches.
What are the patterns for handling database versioning?
Use schema versioning, data versioning. Handle version conflicts. Implement upgrade/downgrade procedures. Consider backward compatibility. Document version dependencies.
How do you implement database connection management?
Use connection pools, proper cleanup. Handle connection lifecycle. Implement health checks. Consider connection timeouts. Handle connection leaks. Monitor connection usage.
What are the best practices for NoSQL database operations?
Handle schema flexibility, denormalization. Implement proper indexing strategies. Consider consistency models. Handle scaling operations. Implement proper error handling. Monitor performance.
How do you handle database configuration management?
Use configuration files, environment variables. Handle different environments. Implement secure credential management. Consider configuration versioning. Document configuration requirements.
What are the strategies for handling database performance tuning?
Monitor query performance, implement indexing. Use query optimization tools. Consider database configuration tuning. Handle connection pooling optimization. Monitor resource usage.
How do you implement database audit logging?
Track data changes, user actions. Implement audit tables. Handle audit log maintenance. Consider compliance requirements. Implement proper retention policies. Handle audit log security.
What are the patterns for handling database failover?
Implement high availability solutions, automatic failover. Handle failover detection, recovery. Consider data consistency. Implement proper monitoring. Document failover procedures.
How do you handle database capacity planning?
Monitor growth trends, resource usage. Plan scaling strategies. Consider performance requirements. Implement monitoring tools. Document capacity requirements. Handle growth projections.
What are the best practices for database maintenance operations?
Schedule maintenance windows, implement automation. Handle index maintenance, statistics updates. Consider performance impact. Implement proper monitoring. Document maintenance procedures.
What are the key differences between Django and Flask frameworks?
Django is a full-featured framework with built-in admin, ORM, auth. Flask is a lightweight, flexible microframework. Django follows 'batteries included' philosophy, while Flask follows minimalist approach. Consider project size, requirements for choice. Django better for large applications, Flask for microservices.
How do you implement RESTful APIs using Flask?
Use Flask-RESTful or Flask API extensions. Implement resource classes, HTTP methods (GET, POST, etc.). Handle serialization, authentication. Consider API versioning, documentation (Swagger/OpenAPI). Implement proper error handling and status codes.
What are Django middleware and how do they work?
Middleware processes requests/responses globally. Implements security, sessions, authentication. Order matters in MIDDLEWARE setting. Can modify request/response objects. Consider performance impact, execution order. Implement custom middleware for cross-cutting concerns.
How do you handle authentication in Python web applications?
Use built-in auth in Django, Flask-Login for Flask. Implement JWT for APIs. Handle session management, password hashing. Consider OAuth integration, multi-factor authentication. Implement proper security measures and token management.
What are the best practices for handling form validation?
Use form classes (Django Forms, WTForms). Implement server-side validation, CSRF protection. Handle file uploads safely. Consider client-side validation, error messages. Implement proper sanitization and validation rules.
How do you implement caching in web applications?
Use cache frameworks (Django's cache, Flask-Caching). Implement view caching, template fragment caching. Consider cache invalidation strategies. Handle cache backends (Redis, Memcached). Implement proper cache key management.
What are the approaches for handling database migrations in Django?
Use makemigrations and migrate commands. Handle schema changes, data migrations. Test migrations before deployment. Consider backwards compatibility. Implement proper rollback procedures. Document migration dependencies.
How do you implement websockets in Python web applications?
Use channels in Django, Flask-SocketIO in Flask. Handle real-time communication, event handling. Consider scaling considerations, connection management. Implement proper error handling and reconnection strategies.
What are the security considerations for Python web applications?
Implement CSRF protection, XSS prevention, SQL injection protection. Use secure headers, HTTPS. Handle input validation, output encoding. Consider security headers, cookie security. Implement proper access controls.
How do you handle file uploads securely?
Validate file types, size limits. Store files securely (filesystem/cloud storage). Handle virus scanning. Consider performance implications. Implement proper cleanup procedures. Handle concurrent uploads.
What are the patterns for API versioning?
Use URL versioning, header versioning, or content negotiation. Handle backwards compatibility. Consider documentation updates. Implement proper version management. Handle deprecated versions gracefully.
How do you implement rate limiting in web applications?
Use rate limiting middleware/decorators. Implement token bucket algorithm. Handle different rate limits per user/API. Consider distributed rate limiting. Implement proper error responses and headers.
What are the strategies for handling background tasks?
Use Celery, Redis Queue, or Django Q. Handle task queuing, scheduling. Implement proper error handling, retries. Consider monitoring, scaling. Handle task priorities and dependencies.
How do you implement user session management?
Use session middleware, handle session storage (database/cache). Consider session timeout, security. Implement proper cleanup. Handle session invalidation. Consider distributed session management.
What are the best practices for error handling in web applications?
Implement proper exception handling, custom error pages. Log errors appropriately. Handle different types of errors (404, 500). Consider user experience. Implement proper error reporting and monitoring.
How do you handle configuration management?
Use environment variables, configuration files. Handle different environments (dev/prod). Implement secure credential management. Consider configuration versioning. Document configuration requirements.
What are the patterns for implementing API documentation?
Use Swagger/OpenAPI specification. Implement automatic documentation generation. Handle versioning, examples. Consider interactive documentation. Implement proper testing of documentation.
How do you implement pagination in web applications?
Use pagination classes (Django) or implement custom pagination. Handle cursor-based, offset pagination. Consider performance implications. Implement proper link headers. Handle edge cases.
What are the strategies for handling CORS?
Implement CORS middleware, handle preflight requests. Configure allowed origins, methods, headers. Consider security implications. Handle credentials properly. Implement proper error responses.
How do you implement testing for web applications?
Use unittest, pytest for testing. Implement unit tests, integration tests. Handle test data, fixtures. Consider test coverage. Implement proper test organization and documentation.
What are the best practices for deployment?
Use containerization (Docker), implement CI/CD. Handle environment configuration. Consider scaling strategies. Implement proper monitoring. Document deployment procedures.
How do you handle static files in production?
Use CDN, proper static file serving. Handle caching, compression. Consider performance optimization. Implement proper cache invalidation. Handle versioning of static files.
What are the patterns for implementing search functionality?
Use search engines (Elasticsearch), implement full-text search. Handle indexing, querying. Consider performance optimization. Implement proper result ranking. Handle search suggestions.
How do you implement logging in web applications?
Use logging framework, handle different log levels. Implement proper log formatting. Consider log aggregation, analysis. Implement proper log rotation. Handle sensitive data in logs.
What are the strategies for handling database connections?
Use connection pooling, handle connection lifecycle. Implement proper error handling. Consider connection timeouts. Handle connection leaks. Monitor connection usage.
How do you implement user roles and permissions?
Use role-based access control (RBAC), implement permission checks. Handle group permissions. Consider hierarchical roles. Implement proper access control lists (ACL).
What are the best practices for template engines?
Use template inheritance, handle template caching. Implement proper escaping. Consider performance optimization. Handle template organization. Implement proper error handling.
How do you handle API rate limiting?
Implement rate limiting middleware, handle quota management. Consider distributed rate limiting. Implement proper headers. Handle burst traffic. Document rate limit policies.
What are the patterns for implementing webhooks?
Handle webhook registration, event delivery. Implement retry logic. Consider security implications. Handle webhook validation. Implement proper error handling.
What are the key differences between NumPy arrays and Python lists?
NumPy arrays are homogeneous (same data type), support vectorized operations, more memory efficient. Offer broadcasting, advanced indexing, mathematical operations. Better performance for numerical computations. Fixed size vs dynamic size of lists.
How do you handle missing data in Pandas?
Use fillna(), dropna(), interpolate() methods. Handle different types of missing data (NaN, None). Consider imputation strategies (mean, median, forward/backward fill). Check missing patterns. Handle missing data in calculations.
What are the different methods for data visualization using matplotlib and seaborn?
Matplotlib for basic plots (line, scatter, bar). Seaborn for statistical visualizations (distributions, regressions). Handle customization, styling. Consider plot types for different data. Implement interactive features.
How do you perform data aggregation in Pandas?
Use groupby(), agg(), pivot_table(). Apply different aggregation functions. Handle multi-level aggregation. Consider performance implications. Implement custom aggregation functions. Handle grouping with different criteria.
What are broadcasting rules in NumPy?
Broadcasting allows operations between arrays of different shapes. Rules: dimensions must be compatible (same, one, or missing). Automatically expands arrays to match shapes. Consider memory implications. Handle dimension compatibility.
How do you handle categorical data encoding?
Use get_dummies() for one-hot encoding, LabelEncoder for label encoding. Handle ordinal vs nominal categories. Consider feature hashing for high cardinality. Implement proper encoding strategy for ML models.
What are the methods for data normalization and scaling?
Use StandardScaler, MinMaxScaler, RobustScaler. Handle outliers in scaling. Consider feature distribution. Implement proper scaling strategy. Handle scaling in train/test split.
How do you handle time series data in Pandas?
Use datetime indexing, resample(), rolling(). Handle time zones, frequencies. Implement time-based operations. Consider seasonal decomposition. Handle missing timestamps. Implement proper date parsing.
What are the techniques for handling imbalanced datasets?
Use SMOTE for oversampling, undersampling techniques. Implement class weights. Consider ensemble methods. Handle evaluation metrics properly. Implement cross-validation strategy for imbalanced data.
How do you perform feature selection in Python?
Use SelectKBest, RFE, feature importance from models. Consider correlation analysis, mutual information. Implement proper validation strategy. Handle feature selection in pipeline.
What are the methods for handling outliers?
Use IQR method, z-score method. Consider domain knowledge for outlier definition. Implement proper outlier treatment strategy. Handle outliers in different features. Consider impact on model performance.
How do you optimize pandas operations for large datasets?
Use chunking, memory efficient methods (read_csv chunks). Consider dtype optimization. Implement proper indexing strategy. Use efficient operations (vectorization). Handle memory constraints.
What are the different methods for data sampling?
Use random sampling, stratified sampling, systematic sampling. Consider sample size, representation. Implement proper sampling strategy. Handle sampling in time series. Consider sampling bias.
How do you handle data merging and concatenation in Pandas?
Use merge(), concat(), join(). Handle different join types. Consider memory implications. Implement proper key matching strategy. Handle duplicates in merging.
What are the techniques for dimensionality reduction?
Use PCA, t-SNE, UMAP. Consider feature importance, correlation. Implement proper validation strategy. Handle scaling before reduction. Consider interpretation of reduced dimensions.
How do you handle text data preprocessing?
Use tokenization, stemming/lemmatization. Handle stop words, special characters. Implement proper text cleaning strategy. Consider language specifics. Handle text encoding issues.
What are the methods for cross-validation?
Use KFold, StratifiedKFold, TimeSeriesSplit. Handle validation strategy selection. Consider data characteristics. Implement proper scoring metrics. Handle cross-validation with parameter tuning.
How do you implement data pipelines using scikit-learn?
Use Pipeline class, FeatureUnion. Handle preprocessing steps. Implement proper transformation order. Consider parameter tuning in pipeline. Handle custom transformers.
What are the techniques for handling multicollinearity?
Use correlation analysis, VIF calculation. Consider feature selection strategies. Implement proper feature elimination. Handle correlation in model building. Consider impact on model interpretation.
How do you perform hypothesis testing in Python?
Use scipy.stats for statistical tests. Handle different test types (t-test, chi-square). Consider assumptions, sample size. Implement proper test selection. Handle multiple testing.
What are the methods for handling data versioning?
Use DVC (Data Version Control), implement proper tracking. Handle dataset versions. Consider storage implications. Implement proper documentation. Handle version dependencies.
How do you optimize NumPy operations?
Use vectorization, proper array operations. Consider memory layout. Implement efficient algorithms. Handle large arrays properly. Consider parallel processing options.
What are the techniques for feature engineering?
Create interaction features, polynomial features. Handle domain-specific transformations. Implement proper feature validation. Consider feature importance. Handle feature scaling.
How do you handle data validation and quality checks?
Implement data validation rules, quality metrics. Handle data integrity checks. Consider domain constraints. Implement proper error handling. Document validation procedures.
What are the methods for handling non-linear relationships?
Use polynomial features, spline transformations. Consider feature transformations. Implement proper validation strategy. Handle overfitting risks. Consider model selection.
How do you implement parallel processing in data operations?
Use multiprocessing, Dask for parallel operations. Handle memory management. Consider scalability issues. Implement proper error handling. Consider overhead vs benefits.
What are the techniques for data augmentation?
Implement different augmentation strategies. Handle domain-specific augmentation. Consider data balance. Implement proper validation strategy. Handle augmentation in pipeline.
How do you handle data streaming and real-time processing?
Use appropriate streaming libraries, implement proper buffering. Handle real-time updates. Consider memory management. Implement proper error handling. Handle data consistency.
What are the methods for model interpretation?
Use SHAP values, feature importance analysis. Implement model-specific interpretation techniques. Consider global vs local interpretation. Handle complex model interpretation.
What is the Global Interpreter Lock (GIL) and how does it affect threading in Python?
GIL is a mutex that protects access to Python objects, preventing multiple native threads from executing Python bytecode simultaneously. Limits CPU-bound threads to run sequentially. Affects only CPython implementation. Consider multiprocessing for CPU-bound tasks.
What are the key differences between threading and multiprocessing in Python?
Threading shares memory space, affected by GIL, good for I/O-bound tasks. Multiprocessing uses separate memory spaces, bypasses GIL, good for CPU-bound tasks. Consider overhead, data sharing needs, and system resources when choosing.
How do you implement asyncio coroutines and event loops?
Use async/await syntax, create event loop using asyncio.get_event_loop(). Handle coroutine execution, task scheduling. Consider error handling, cancellation. Implement proper cleanup. Handle event loop lifecycle.
What synchronization primitives are available in Python's threading module?
Lock, RLock (reentrant lock), Semaphore, Event, Condition, Barrier. Each serves different synchronization needs. Handle proper acquisition/release. Consider deadlock prevention. Implement proper error handling.
How do you share data between processes safely?
Use multiprocessing.Queue, Pipe, shared memory (Value, Array). Handle synchronization properly. Consider serialization overhead. Implement proper locking mechanisms. Handle process termination cleanup.
What is the difference between asyncio.gather() and asyncio.wait()?
gather() returns results in order, raises first exception. wait() provides more control over completion, timeout, returns done/pending sets. Consider exception handling, cancellation behavior. Handle task dependencies.
How do you implement thread pools in Python?
Use concurrent.futures.ThreadPoolExecutor. Handle task submission, result collection. Implement proper shutdown. Consider pool size optimization. Handle exceptions in worker threads. Manage resource usage.
What are race conditions and how do you prevent them?
Race conditions occur when multiple threads access shared data simultaneously. Prevent using locks, atomic operations. Implement proper synchronization. Consider thread safety in design. Use thread-safe data structures.
How do you handle deadlocks in multithreaded applications?
Prevent using lock ordering, timeouts. Detect using deadlock detection algorithms. Implement recovery mechanisms. Consider lock hierarchy. Handle nested locks carefully. Implement proper error recovery.
What is the purpose of the async with and async for statements?
async with handles asynchronous context managers. async for iterates over async iterables. Properly handle resource cleanup, iteration. Consider exception handling. Implement proper asynchronous patterns.
How do you implement process pools in Python?
Use multiprocessing.Pool or concurrent.futures.ProcessPoolExecutor. Handle task distribution, result collection. Consider process communication overhead. Implement proper cleanup. Handle process failures.
What are the best practices for handling exceptions in concurrent code?
Implement proper exception handling in worker threads/processes. Handle task failures gracefully. Consider exception propagation. Implement logging and monitoring. Handle cleanup in error cases.
How do you implement thread-safe data structures?
Use Queue from queue module, implement proper locking. Consider atomic operations. Use thread-safe collections. Implement proper synchronization mechanisms. Handle concurrent access patterns.
What is the difference between CPU-bound and I/O-bound tasks?
CPU-bound tasks primarily use processor (calculations). I/O-bound tasks wait for external operations (network, disk). Choose appropriate concurrency model (multiprocessing vs threading). Consider resource utilization.
How do you implement asynchronous context managers?
Implement __aenter__ and __aexit__ methods. Handle async resource acquisition/release. Consider exception handling. Implement proper cleanup. Handle async initialization.
What are the patterns for implementing producer-consumer problems?
Use Queue for thread-safe communication. Implement proper synchronization. Handle termination conditions. Consider buffer size. Implement proper error handling. Handle backpressure.
How do you handle task cancellation in asyncio?
Use Task.cancel(), handle CancelledError. Implement cleanup handlers. Consider task dependencies. Handle cancellation propagation. Implement proper state cleanup.
What are the considerations for thread safety in Python?
Consider GIL implications, use thread-safe operations. Implement proper synchronization. Use atomic operations when possible. Consider thread-local storage. Handle shared resource access.
How do you implement parallel processing for data operations?
Use multiprocessing pools, handle data partitioning. Consider communication overhead. Implement proper merging strategy. Handle process synchronization. Consider memory usage.
What are the patterns for handling timeouts in concurrent operations?
Use timeout parameters, implement timeout handling. Consider cancellation. Handle cleanup after timeout. Implement proper error reporting. Consider partial results handling.
How do you debug concurrent programs effectively?
Use logging, debugger support for threads/processes. Implement proper monitoring. Consider race condition detection. Handle debugging synchronization issues. Implement proper error tracking.
What is the purpose of thread-local storage?
Thread-local storage provides thread-specific data storage. Prevents data sharing between threads. Implement using threading.local(). Consider cleanup requirements. Handle initialization properly.
How do you implement asynchronous generators?
Use async def with yield, implement __aiter__ and __anext__. Handle async iteration properly. Consider resource management. Implement proper cleanup. Handle cancellation.
What are the best practices for process pool management?
Handle pool lifecycle, implement proper shutdown. Consider resource cleanup. Handle worker process errors. Implement proper task distribution. Consider pool size optimization.
How do you handle interprocess communication effectively?
Use appropriate IPC mechanisms (Queue, Pipe). Handle synchronization properly. Consider serialization overhead. Implement proper error handling. Handle process termination.
What are the patterns for implementing event-driven programming?
Use event loops, implement event handlers. Consider event propagation. Handle event ordering. Implement proper error handling. Consider event queue management.
How do you optimize concurrent operations performance?
Balance thread/process count, optimize resource usage. Consider overhead costs. Implement proper task granularity. Handle load balancing. Consider system resources.
What are the considerations for scaling concurrent applications?
Handle resource limitations, implement proper load balancing. Consider scalability bottlenecks. Handle distributed processing. Implement proper monitoring. Consider performance metrics.
How do you implement graceful shutdown in concurrent applications?
Handle termination signals, implement proper cleanup. Consider ongoing operations. Handle resource release. Implement proper state saving. Consider recovery procedures.
What are the key principles of PEP 8 and why are they important?
PEP 8 defines Python's style guide: indentation (4 spaces), line length (79 chars), naming conventions (snake_case for functions/variables, PascalCase for classes), import organization. Ensures code readability and consistency across Python community. Key for maintainability and collaboration.
What are Python's SOLID principles and how are they implemented?
Single Responsibility (one purpose per class), Open/Closed (open for extension, closed for modification), Liskov Substitution (subtypes must be substitutable), Interface Segregation (specific interfaces), Dependency Inversion (depend on abstractions). Implement using proper class design, inheritance, and abstractions.
How do you write effective docstrings?
Follow PEP 257: describe purpose, parameters, return values, exceptions. Use consistent format (Google, NumPy, reStructuredText). Include examples when appropriate. Consider doctest integration. Keep updated with code changes. Essential for code understanding and maintenance.
What are the best practices for exception handling?
Catch specific exceptions, not bare except. Use context managers for resource cleanup. Implement proper error hierarchy. Log exceptions appropriately. Consider error recovery strategies. Don't suppress exceptions without good reason. Document exceptional cases.
How do you implement the DRY (Don't Repeat Yourself) principle?
Extract common code into functions/classes. Use inheritance and composition appropriately. Implement utility modules. Consider code reusability. Balance DRY with readability. Use proper abstraction levels. Consider maintenance implications.
What are the patterns for dependency injection in Python?
Pass dependencies as parameters, use dependency containers, implement factory patterns. Consider interface-based design. Handle dependency lifecycle. Use proper abstraction. Consider testing implications. Implement proper configuration management.
How do you organize a Python project structure?
Use proper package hierarchy, separate concerns, implement clear imports. Follow standard project layout (setup.py, requirements.txt, docs/, tests/). Consider module organization. Implement proper configuration management. Document project structure.
What are the principles of clean code in function design?
Functions should be small, do one thing, have clear names. Use proper parameter design, avoid side effects. Consider return value clarity. Implement proper validation. Document function behavior. Consider error cases.
How do you implement proper logging practices?
Use appropriate log levels, implement structured logging. Consider log format, destination. Handle sensitive data appropriately. Implement proper error reporting. Consider log rotation, retention. Document logging conventions.
What are the best practices for code reviews?
Check style consistency, proper error handling, test coverage. Review documentation, performance implications. Consider security aspects. Look for code smells. Provide constructive feedback. Document review process.
How do you handle configuration management properly?
Use configuration files, environment variables. Implement proper validation. Handle different environments. Consider security implications. Document configuration options. Implement proper defaults.
What are the patterns for implementing interfaces in Python?
Use abstract base classes (ABC), protocol classes. Implement proper method signatures. Consider duck typing. Handle interface evolution. Document interface contracts. Consider backward compatibility.
How do you write maintainable unit tests?
Follow AAA pattern (Arrange-Act-Assert), use clear test names. Implement proper test isolation. Consider test readability. Handle test data properly. Document test cases. Maintain test suite.
What are the best practices for code comments?
Comment why, not what. Keep comments updated with code. Use proper inline documentation. Consider self-documenting code. Document complex algorithms. Avoid redundant comments. Keep comments concise.
How do you implement proper error messages?
Make messages clear, actionable. Include relevant context. Consider user perspective. Handle internationalization. Document error conditions. Implement proper error categorization. Consider error recovery.
What are the principles of package design?
Follow single responsibility, maintain clear interfaces. Consider versioning strategy. Handle dependencies properly. Document package usage. Implement proper testing. Consider distribution aspects.
How do you handle code deprecation?
Use deprecation warnings, provide migration path. Document deprecation timeline. Maintain backward compatibility. Consider impact on users. Implement proper versioning. Handle removal process.
What are the patterns for implementing factory methods?
Use factory methods for object creation, implement proper initialization. Consider inheritance hierarchy. Handle configuration. Document factory behavior. Consider error cases.
How do you implement proper variable naming?
Use descriptive names, follow conventions. Consider scope visibility. Use proper prefixes/suffixes. Maintain consistency. Document naming patterns. Consider international aspects.
What are the best practices for code optimization?
Profile before optimizing, focus on bottlenecks. Consider algorithmic efficiency. Use appropriate data structures. Handle memory usage. Document optimization decisions. Consider maintenance implications.
How do you implement proper class design?
Follow single responsibility, use proper inheritance. Implement encapsulation. Consider composition over inheritance. Document class behavior. Handle initialization properly. Consider class evolution.
What are the patterns for handling global state?
Minimize global state, use dependency injection. Consider thread safety. Implement proper access patterns. Document global dependencies. Consider testing implications. Handle state initialization.
How do you implement proper module organization?
Group related functionality, maintain clear interfaces. Consider circular dependencies. Document module purpose. Implement proper importing. Handle module initialization. Consider module evolution.
What are the best practices for API design?
Keep interfaces simple, document clearly. Handle versioning properly. Consider backward compatibility. Implement proper validation. Document API changes. Consider security aspects.
How do you handle code complexity?
Break down complex functions, use proper abstraction. Implement clear control flow. Consider cognitive complexity. Document complex logic. Use appropriate design patterns. Consider maintenance aspects.
What are the patterns for implementing decorators?
Use proper wrapper functions, maintain function metadata. Handle arguments properly. Consider chaining decorators. Document decorator behavior. Consider performance implications.
How do you implement proper error recovery?
Implement proper cleanup, handle partial failures. Consider system state. Document recovery procedures. Implement proper logging. Handle cascading failures. Consider user experience.
What are the best practices for code documentation?
Use clear documentation style, keep documentation updated. Consider audience needs. Document assumptions. Implement proper examples. Consider automation tools. Maintain documentation quality.
How do you implement proper version control practices?
Use meaningful commit messages, proper branching strategy. Consider merge management. Document version changes. Implement proper tagging. Handle release process. Consider collaboration aspects.
What are metaclasses in Python and when should they be used?
Metaclasses are classes for classes, allowing customization of class creation. Used for API enforcement, attribute validation, class registration, abstract base classes. Created using type or custom metaclass. Example: class MyMetaclass(type): def __new__(cls, name, bases, attrs). Consider complexity trade-offs.
How do descriptors work and what are their use cases?
Descriptors define __get__, __set__, __delete__ methods to customize attribute access. Used in properties, methods, class attributes. Enable managed attributes, validation, computed values. Example: property implementation using descriptors. Consider performance implications.
What is the difference between __new__ and __init__?
__new__ handles object creation (called before __init__), returns instance. __init__ initializes instance (called after __new__). __new__ used for immutable types, singletons, metaclasses. __init__ can't return value. Consider inheritance implications.
How does Python's garbage collection work with reference counting and cycle detection?
Uses reference counting for basic GC, generational GC for cycles. Reference count tracks object references, frees when zero. Cycle detector identifies and collects unreachable reference cycles. Consider memory management, performance implications.
What are abstract base classes and how do they differ from interfaces?
ABCs define interface through @abstractmethod, can include implementation. Created using abc.ABC. Enforce interface implementation, provide default methods. Unlike Java interfaces, can have implementation. Consider multiple inheritance implications.
How do you implement custom context managers?
Implement __enter__ and __exit__ methods, or use contextlib.contextmanager decorator. Handle resource acquisition/release, exception handling. Consider cleanup guarantees. Example: file handling, database connections.
What are the advanced features of Python generators?
send() method for two-way communication, throw() for exception injection, close() for generator cleanup. Generator delegation using yield from. Consider coroutine behavior, exception handling. Implement proper cleanup.
How do you implement custom attribute access?
Use __getattr__, __setattr__, __getattribute__, __delattr__. Handle attribute lookup, modification, deletion. Consider infinite recursion, performance. Implement proper attribute management. Handle special cases.
What is method resolution order (MRO) and how does it work?
MRO determines method lookup order in inheritance. Uses C3 linearization algorithm. Accessible via __mro__ attribute. Handles multiple inheritance, method overriding. Consider inheritance complexity, diamond problem.
How do you implement custom iteration protocols?
Implement __iter__ and __next__ methods for iteration. Consider StopIteration handling. Implement proper cleanup. Handle resource management. Consider memory efficiency. Example: custom sequence types.
What are slots and when should they be used?
__slots__ restricts instance attributes, reduces memory usage. Faster attribute access, prevents dynamic attribute addition. Consider inheritance implications, flexibility trade-offs. Use for memory-critical applications.
How do you implement custom comparison methods?
Implement rich comparison methods (__eq__, __lt__, etc.). Use @functools.total_ordering for complete ordering. Consider type checking, reflexivity, transitivity. Handle edge cases. Implement proper comparison logic.
What is the difference between bound and unbound methods?
Bound methods are instance-specific, include self parameter automatically. Unbound methods are class-level, require explicit instance. Consider method types, descriptor protocol. Handle method binding properly.
How do you implement custom pickling behavior?
Implement __getstate__ and __setstate__ for custom serialization. Handle complex objects, circular references. Consider security implications. Implement proper state management. Handle version compatibility.
What are type hints and how do they enhance code?
Type hints provide static typing information. Used by type checkers (mypy), IDE support. Not enforced at runtime by default. Enhance code readability, maintainability. Consider performance impact, compatibility.
How do you implement custom container types?
Implement container protocol methods (__len__, __getitem__, etc.). Consider sequence/mapping behavior. Handle iteration, containment checks. Implement proper indexing. Consider memory management.
What are class decorators and how do they differ from function decorators?
Class decorators modify/enhance class definitions. Applied using @decorator syntax above class. Can modify class attributes, methods, add functionality. Consider inheritance implications. Handle class modification properly.
How do you implement custom exception classes?
Inherit from appropriate exception base class. Implement proper initialization, custom attributes. Consider exception hierarchy. Handle error messages, context. Implement proper cleanup. Document exception conditions.
What are protocols in Python and how do they work?
Protocols define interface behavior without inheritance. Use typing.Protocol for static typing. Support structural subtyping. Consider duck typing implications. Implement protocol compatibility.
How do you optimize memory usage in Python?
Use __slots__, proper data structures, generators. Consider memory profiling, garbage collection. Handle large datasets efficiently. Implement proper cleanup. Monitor memory usage. Consider caching strategies.
What are assignment expressions (walrus operator) and their use cases?
Assignment expressions (:=) assign and return value in single expression. Used in while loops, if statements, list comprehensions. Enhances code readability, reduces redundancy. Consider proper usage contexts.
How do you implement custom number types?
Implement numeric protocol methods (__add__, __mul__, etc.). Handle reverse operations (__radd__, etc.). Consider type coercion, precision. Implement proper arithmetic behavior. Handle special cases.
What are generics in Python and how are they used?
Generics provide type hints for collections/classes. Use typing.Generic, TypeVar. Support parametric polymorphism. Consider type checking implications. Implement proper type constraints. Handle type variables.
How do you implement lazy property evaluation?
Use @property with caching, implement custom descriptors. Consider computation cost, memory usage. Handle invalidation properly. Implement thread safety. Consider cache management.
What are dataclasses and their advanced features?
@dataclass decorator provides automatic __init__, __repr__, etc. Support inheritance, frozen instances, default values. Consider field options, ordering. Implement custom initialization. Handle field dependencies.
How do you implement custom function and method wrappers?
Use decorator pattern, functools.wraps for metadata preservation. Handle argument passing, return values. Consider function attributes. Implement proper wrapping. Handle special methods.
What are async and await magic methods?
__aiter__, __anext__, __aenter__, __aexit__ for async operations. Support async context managers, iterators. Handle coroutine behavior. Implement proper cleanup. Consider async patterns.
How do you implement custom attribute descriptors?
Create descriptor classes with __get__, __set__, __delete__. Handle attribute access, modification. Consider validation, computation. Implement proper state management. Handle inheritance.
What are the advanced features of Python's memory model?
Understand object internals, reference counting, memory allocation. Handle weak references, object lifecycle. Consider memory sharing, copy semantics. Implement proper memory management. Handle circular references.
Stark.ai
Instant Interviews
