I'm auditing a project that uses what is called a Rules Engine. In short, it's a way to externalize business logic from application code.
This concept is entirely new to me and I'm pretty skeptical about it. After hearing people talk about Anemic Domain Models for the past few years, I'm questioning the Rules Engine Approach. To me they seem like a great way to WEAKEN a domain model. For example say I'm doing a java webapp interacting with a Rules Engine. Then I decide I want to have an Android app based on the same domain. Unless I want the Android app to interact with the Rules Engine as well, I'm going to have to miss out on whatever business logic was already written.
As I don't have any experience with them yet, just curiosity, I was interested to hear about the pros and cons are in using a Rules Engine? The only pro that I can think of is that you don't need to rebuild your entire Application just to change some business rule (but really, how many apps really have that many changes?). But using a Rules Engine to solve that problem kind of sounds to me like putting a band-aid over a shotgun wound.
UPDATE - since writing this, the god himself, Martin Fowler, has blogged about using a Rules engine.
Answer
Most rule engines that I have seen are viewed as a black box by system code. If I were to build a domain model, I would probably want certain business rules to be intrinsic to the domain model, e.g. business rules that tell me when an object has invalid values. This allows multiple systems to share the domain model without duplicating business logic. I could have each system use the same rule service to validate my domain model, but this appears to weaken my domain model (as was pointed out in the question). Why? Because instead of consistently enforcing my business rules across all systems at all times, I am relying on system programmers to determine when the business rules should be enforced (by calling the rule service). This may not be a problem if the domain model comes to you completely populated, but can be problematic if you're dealing with a user interface or system that changes values in the domain model over its lifetime.
There is another class of business rules: decision making. For example, an insurance company may need to classify the risk of underwriting an applicant and arrive at a premium. You could place these types of business rules in your domain model, but a centralized decision for scenarios like this are usually desirable and, actually, fit quite well into a service-oriented architecture. This does beg the question of why a rule engine and not system code. The place where a rule engine may be a better choice is where business rules responsible for the decision change over time (as some other answers have pointed out).
Rule engines usually allow you to change rules without restarting your system or deploying new executable code (regardless of what promises you receive from a vendor, do make sure you test your changes in a non-production environment because, even if the rule engine is flawless, humans are still changing the rules). If you're thinking, "I can do that by using a database to store values that change", you're right. A rule engine is not a magical box that does something new . It is intended to be a tool that provides a higher level of abstraction so you can focus less on reinventing the wheel. Many vendors take this a step further by letting you create templates so that business users can fill in the blanks instead of learning a rule language.
One parting caution about templates: templates can never take less time than writing a rule without a template because the template must, at the bare minimum, describe the rule. Plan for a higher initial cost (the same as if you were to build a system that used a database to store values that change vs. writing the rules in directly in system code) - the ROI is because you save on future maintenance of system code.